Tuesday, May 14, 2013

Notes from a RCT guinea pig

I hope you all had a wonderful mothers' day.  I opened mothers' day while madly writing responses to  questions for a CopyrightX final exam.  Bad Dad went to Trader Joe's for a food run and brought me some flowers to liven up the computer desk and chocolates for caffeine and energy.

Much has been written about MOOCs, massive open online classes, but mostly by nontechnical writers.  I think that MOOCs can be wonderful for computer programming courses.  But, they are lacking for humanities and other open-ended disciplines.  If reporters only sample the nontechnical MOOCs, then they will come away with the impression that MOOCs are just an educational fad with no real substance behind it.

Nathan Heller's long article for the New Yorker,  Laptop U, is representative of the coverage Bad Dad and I have read.  He opens with The Ancient Greek Hero, a class that Bad Dad is taking very seriously and I have all but given up on, and closes with CopyrightX, a very rigorous class I just completed.

CopyrightX is not a MOOC in the sense that it is not truly open.  Potential students apply and are selected.  I'll refer to it as a MOC, though, at 500 students, it's neither open nor massive.

We also pledge that, if selected, we will put in at least 8 hours a week of study and discussion time.  To further weed out dilettantes, the online application included several essay questions about our backgrounds and motivations for studying copyright.  I wrote about my past work in data collection and curation and my sewing and blogging hobbies.  My answers were compelling enough to land me one of the 500 internet student slots.

Heller wrote: "In picking students, [professor Terry Fisher] looked partly for a range of ages and professions: his goal is to seed knowledge of digital-age copyright law among people who will apply it creatively in their own circles and work."

 Of the 25 people in my section, three held PhDs, one two were practicing IP attorneys and a handful worked as librarians and archivists for a range of federal and state agencies or nonprofits. This was not your average (wo)man on the street cohort.  Students came from Europe, North and South America and China, but the North American and European students were most active in discussion boards and at the "live" weekly recitation sections.  I'm curious to learn how many of us took the final (which is a proxy for sticking it out in the class).

In addition to the weekly lectures by the professor, we read review articles and case law, "met" with our section for weekly discussion with our TA as a moderator and watched biweekly guest lectures.  Our TA also offered weekly office hours and sent out additional study materials throughout the weeks. If we asked a question that our TA couldn't answer, she sent it out to her fellow TAs and the professor and replies rolled in reasonably quickly.

Functionally, CopyrightX was a small college seminar, taught by a graduate teaching assistant and overseen by a rock-star professor.  The TA was super generous with her time.  The professor, though not approachable directly by students, spent considerable time preparing the class and listening to the TAs and the students through the TAs.

The internet 500 (CopyrightX group) took the class at the same time as students enrolled in the bricks and mortars Harvard Law School (HLS), studying the same units at the same pace.  In discussion section, HLS and CopyrightX students discussed the same "hypotheticals", or hypothetical cases.  During the special guest seminars, both HLS and CopyrightX students could submit questions.  Other students and TAs would upvote questions on the board, and the guests answered the top-voted questions from both groups.

The CopyrightX welcome message mentioned that we were assigned to one of four groups.  I instantly wrote my teaching assistant to ask if we were taking part in a randomized clinical trial (RCT).  (I learned about RCTs in PH207x Health in Numbers: Quantitative Methods in Clinical & Public Health Research.)

My TA asked the edX staff and a response rolled in many weeks later as the question got kicked around until it found someone who knew the methodology.  I know that you are on the edge of your seat, waiting for the methodology.  Here goes.

Each of the 500 admitted and enrolled students checked off one or more of the five available time slots.  The computer assigned 100 to each time slot, based on our availability.  The 20 TAs were assigned, four to a time slot, in a similar fashion.  Then we were randomly assigned, within each time slot, to one of four groups.

There were two "exposures" and four possible combinations.  One half read US case law only (controls) while the other read fewer US case law but more international law.  One half used the edX discussion boards only (controls) while the other half also used Nota Bene PDF annotation software to collaboratively mark up the weekly readings.  I was assigned to the "double exposure" group, which studied international copyright law and used Nota Bene.  (If you haven't tried Nota Bene, give it a whirl.  I really like it.)

Come to think of it, there is a third exposure--online or bricks and mortar.  In one section, the TA let it slip that we came to consensus on the relevant section of law and precedents faster than the HLS section.  That's when I realized that they could compare us to not just other CopyrightX students, but also possibly to the HLS students.

In early June, we will be notified of our final exam results.  Undoubtedly, the performance data between groups will provide researchers with rich material.  I hope that our participation as guinea pigs will make online education better for all.

I learned as much from my fellow students as from the TA. The accreditation doesn't matter to someone in my situation (though the librarians and archivists were taking the class as part of their jobs).  Being in a very serious cohort with breadth and depth of real world experience and with true interactivity were valuable to me.   I would pay money for such courses once they move out of clinical trial--if the RCTs prove MOC efficacy.  ;-)

That said, I initially put a blanket copyright notice at the bottom of this blog because I didn't want to give away any rights before I understood them.  Now that I am more aware, I think I will change the copyright to Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported as a baseline and then put separate notices for content that I want to protect differently.

I'll also post later about what the licenses mean and how I will apply them to my blogging, sewing, knitting, software and data activities.  Stay tuned.  Watch this footer.  ;-)

Leave a comment if you think this is interesting or I should just stick to the sewing and knitting stuff.


Commentator Douglas Kretzman left two excellent links.   The MOOC Moment and the End of Reform By Aaron Bady is the best discussion of what MOOCs means for education in particular and society in general I have ever read. It is truly not to be missed.

Then read this tongue in cheek but deadly serious CUCFA President Meister's Open Letter to Coursera Founder Daphne Koller.


  1. Really interesting! I love your varied posts and this is an area we all need to know about now. I'm impressed - was this class free?

  2. Interesting. Thanks for sharing!

  3. Yes, the class was free. We were beta testers in addition to guinea pigs.

    They may charge in the future, but edX classes are all free right now. There is talk of monetizing classes in the future.

    Given the huge tax breaks that private colleges enjoy, and the billions and billions in their endowment funds that grow tax-free, some people feel that MOOCs are the least they can do for the public.

    Nonprofits are supposed to do something for the public for their tax exempt status. Congress had been sniffing around the endowment funds of the elite universities and there is the whiff of corruption.

    It's a long and complicated story. The gist of it is that the elite universities feel some heat to spend ~5% of their endowment funds annually on activities for the public benefit. $60 million over two years from MIT and Harvard combined is a drop in the bucket for them.

  4. this was fascinating, thank you.
    I put my sparse blog on the same CC license some time ago..

    what scares me about MOOCs is that many of them are venture-capital owned. These are profit-seeking missiles, I fear the collateral damage to old-fashioned teaching. See for example

  5. Very interesting, especially the research protocol.
    I like that your blog is a mix of things that I am also interested in, like sewing, science, and family.

  6. Yes, this IS very interesting. I really enjoy your varied posts as well.

  7. I've sent the link to this post to my department head, since MOOS and their pros and cons for education is something we've just discussed! Thank you for sharing!

  8. This is so interesting badmomgood mom!

    I want to thank you specifically for your very useful post on blog theft too... that you provided the link in my comments section. I have only just got around to reading it properly and it is, as is usual for your posts, extremely thorough, well written and helpful. I hope you don't mind if I link to your post through my blog?
    You are a goldmine of useful and relevant information :)
    Thank you so much.

  9. also on MOOCs, zunguzungu as always repays reading:


  10. @DK Thank-you for the two EXCELLENT links. I will add them to the post as well. Aaron Bady's analysis in zunguzungu is the best I've seen and not to be missed.

  11. Very interesting. I am loving my MOOC Greek Hero Class (but remember, I was an Ancient History major, minor in Biology at Berkeley).

  12. @LHC That's precisely the point of Bady's article. You are already a graduate of one of the most prestigious Classics departments in the world. It's a good refresher and enrichment activity for you. But, what value does it have for others without your background?

  13. I was wondering that same thing. I get it, because I was already trained to understand all the concepts. A kid just out of high school would be at sea without a actual class with a live human teacher. My son read the Iliad at the same time in his history of economics class and we discussed the economy of Mycenean Greece. But I don't see how this can work for college freshmen.

  14. Anonymous09:17


    I really enjoyed reading your post.
    I am a law student from India and have been selected for the CopyrightX course this year.
    I'd be highly grateful if you would be so kind as to share your outline with me.
    Even though I am preparing my own outline, your notes will certainly help me in acquiring a better understanding of the course material.
    My email ID is rahul.bajaj1038@gmail.com
    I look forward to hearing from you.



Comments are open for recent posts, but require moderation for posts older than 14 days.