A few weeks ago I was asked to give a short presentation to my department about my research. My slides are below, both as a slideshare and also as a PDF with commentary notes.

The presentation gives a brief overview of why data sharing is important, why we need to study it, some of what we’ve learned, and what we still don’t know. Spoiler: it concludes with an encouragement to share your own research data.

Why study Data Sharing? (+ why share your data)

click for Slides with Commentary Notes

Two process questions:

1. Does anyone know of a better way to share slide spoken content, other than “PDF with notes”? I don’t *want* my slides to include everything I say, because that makes for a boring presentation in real life. But how else to communicate the information when archiving? Slideshare does allow voice-over, but frankly that raises the barrier of entry for me (record a podcast, aack). Any other ideas?

2. Any open clip art sources, other than http://www.openclipart.org/ and http://www.wpclipart.com/ ? These have lots of things, but unfort still don’t rival the variety that comes with PowerPoint. I looked, but there just didn’t include the right hats.  So the presentation includes Microsoft clipart. Guessing that means I can’t make those images CC. Guessing that means I can’t post the presentation on Nature Precedings. Oh well, small steps.

Thanks to Klaus Graf for putting it so succinctly (and PMR for echoing it):

There is a simple solution (I will repeat it because it is important like a mantra):

* MAKE ALL RESEARCH RESULTS CC-BY
* MAKE ALL RESEARCH RESULTS CC-BY
* MAKE ALL RESEARCH RESULTS CC-BY

Make all research results CC-by.

Really. The publishers, yes. But also You. Me. All of us. Try it. Start small, ok, but start. Put your raw data up on a website and slap a Creative Commons license on it.

Be the change you want to see.

Interesting discussion on Peter Murray-Rust’s blog about whether PubMed Central articles can be crawled and used for text mining. The answer is no, not now, not unless they are open access (as opposed to traditional closed access but deposited in PMC).  Really unfortunate.  Incremental progress, we’ll get there.

Anticipating my thesis work, I’ve been wondering about similar text mining questions. I think my needs are a bit different than those of PMR: I’m interested in papers that meet a targeted search, rather than all articles or all articles in relevant journal (what I gather he’s be interested in?). I’m willing to limit myself to the articles that I have access to through my University’s subscriptions. I don’t need figures. I think once I have the papers I’m allowed to text mine them as fair use, since I have them under permission. So the question is what can I automatically download?

I learned I can’t spider PMC, but what about normal PubMed? Try as I might, I couldn’t find verbage on the PubMed website allowing/disallowing spidering through to full-text links on publisher websites (the links that are populated and visible when I’m logged in through the University’s connection). Is this allowed? Still seems like it might not be. And then you end up at the publisher sites anyway, with all of their differing rules. Unfort, the publisher’s rules are often hard to find, confusing, and vague (as often noted by PMR and others). Aaaaah.

So last month I asked our librarians….

As you know, PMC has OA and non-OA full-text. They make their OA text available via FTP etc, and they stipulate that those mechanisms are the only way that people are allowed to access the full text “because of copyright restrictions” [http://www.pubmedcentral.nih.gov/about/copyright.html]. I’d also like to access non-OA text for which Pitt has subscriptions, but it sounds like I can’t do this by “crawling” PMC based on their rules [explicitly stated in the link above]. I guess I’m wondering if I can do it by “crawling” the normal, full PubMed. Basically write a script to find the “HSLS” links on the article citation pages, follow them (usually into the publisher’s websites), and automatically save the html or pdf articles that are returned from a PubMed query.

There is no difference in the end result from me manually clicking through and saving the papers… but there is sure a difference in the manual time requirement! I wouldn’t have thought this sort of automated downloading would be a problem… but the Restrictions on Systematic Downloading of articles in the PMC copyright notice referenced above makes me want to double-check. I can’t find any reference to “crawling” or “systematic downloads” for PubMed itself.

I do understand there are user requirements when using the Entrez programming utilities (run automated queries during off hours, 3 seconds between queries, etc) and I would be sure to honor those both with the elements of my scripts which use the E-tools and those which are crawling the web pages directly.

Does that make sense? Are you aware of any restrictions for crawling PubMed to automatically access and save content for which I do indeed have access through Pitt? I guess since I’m going into the publisher’s websites, they might also have restrictions? Is there another way to consolidate a large set electronic full-text articles (ideally a few thousand)?

Thanks very much for any pointers you may have.

The librarian responsed that automatically following PubMed links should be fine, and that there shouldn’t be problems from publisher sites because we have subscriptions and my text mining falls under fair use. I’ll add that I think it helps that I’m not aiming to download full editions, because I do know that some publisher websites disallow that.

Maybe I shouldn’t be bringing it up again here, since it feels like I’ve been given an institutional “All Clear.” But no sense burying my head in the sand in case there really are issues: I want to know. Web downloading policies and full-text reuse policies are so complicated. I’ve spent time looked into them, but it sure seems like unless it is your full-time job it is impossible to understand and keep on top of how it works. I don’t think our librarians deal with these issues every day. Who else would I go to for clarification?

Does anyone have differing interpretations, warnings, reassurances, alternatives, and general paths through this crazy mess? How do other people do this???

Thanks to Cameron Neylon for describing what friendfeed is all about.  It looks neat, so I’m giving it a go. There isn’t an explicit way to add a “citeulike” source to monitor the papers I’m archiving, but I’ve added my citeulike username page as a “blog” and that seems to work.  For now I’m keeping my sources worky (ie not linking to the Flickr baby photostream) to keep the work-signal high.  Ok, except I did include my starred YouTube videos… low traffic, usually nerdy, and some of that stuff is way too funny not to pass on :)

Friendfeed addresses something I’ve been struggling with.  Ideally, I want to blog about the things I read that I find particularly interesting and relevant, to make sure that everyone else has seen them too (ok, they probably have, but just in case…).  Needless to say, I’m not getting to it.  I need to let it go.  So in case you’re interested in the things I’m interested in, have a look at my friendfeed.  And introduce yourself or leave a link to your blog/library/paper stream/something so I can follow your work too!

I actively support Nature Precedings as a place to archive my early research findings for visibility, feedback, and attribution. I recently submitted a research proposal. Precedings does a spot-check of all submissions to verify appropriateness. It usually takes a day or two, and results in an automated response stating that your submission has been posted. In this case, I received a personal, thoughtful email explaining that although Nature Precedings had published proposals in the past, they are moving away from this practice to concentrate on their core goal of “a repository for manuscripts, posters, and presentations describing completed research.”

I see the issue. On one hand, if the goal of a preprint is to get feedback and attribution for research ideas, what better time to do it than at the proposal stage. On the other hand, ideas are a dime a dozen and so it might not be scalable for Nature Precedings.

Sounds like we need another solution. There was a letter to Nature just recently (highlighted by Maxine Clarke in Nautilus), calling for a Centralized Proposal Repository. This idea is grander than simply a wiki for feedback and attribution. Dr Harel is suggesting it could also be searched by funders, to identify projects which match their interests.

Dr Harel expands on this idea on his website and links to the beginning of a Centralized Proposal Repository wiki.

Great idea, and I think right up the alley of open science. The wiki seems to be Protected (to limit spammers?) I’ll go ask to join, and keep you updated on what I learn.

ETA: Looks like Jean-Claude Bradley was involved in the set-up of the wiki. Great stuff!

Does anyone remember reading a paper about the citation history of ape studies? If so, I’d love a pointer or any keyword-details you might remember. I haven’t been able to find it.

For those who are curious: from what I remember, the authors were investigating the what proportion of ape (or was it monkey) studies are truly necessary. They collected citation histories and discovered that a large proportion of studies had received no citations at all. This suggests that although such studies have to clear high hurdles to be approved, our mechanisms are insufficient and as a result we’re putting our cousins through unnecessary hardship.

Interesting to me on a number of levels. 1. It would be a helpful reference for the usage of citation histories. 2. We’ve been to the zoo recently with a gorilla-enthralled two year old.  The apes (and monkeys) act just like some people I know.  Any invasive research (or zookeeping…) had better be worth it.

As reported by the organizers at One Big Lab and Science in the Open, PSB 2009 is going to have a 3 hour workshop devoted to Open Science. Neat, eh? What a great chance to meet and learn from others who are working this way and/or thinking about this topic! And discuss it with others who just happen to be in the PSB neighbourhood! And go to Hawaii in January! :)

I’ll definitely be submitting a talk proposal. Still brainstorming the topic. The winner in my head over the last 24 hours is “Open Science: Measuring the Costs and Benefits.” What do you think? Other ideas? Off to email Shirley and Cameron…..

I emailed AMIA again to ask for clarification on their preprint policy, and quickly received this encouraging response: “Preposting is fine so long as the other sites don’t formally publish the work.” Great news, thanks AMIA.

Note: this brings my blog up-to-date on the research I’ve been doing, with the exception of one paper under review at PLoS Medicine. That one is a complex collaboration. Despite some attempts there isn’t consensus about making it open at this point.

Here is the paper we submitted to the AMIA 2008 Annual Symposium. AMIA=American Medical Informatics Association. Nature Precedings link to appear once it has been posted.

Identifying Data Sharing in Biomedical Literature
Heather A. Piwowar and Wendy W. Chapman

Many policies and projects now encourage investigators to share their raw research data with other scientists. Unfortunately, it is difficult to measure the effectiveness of these initiatives because data can be shared in such a variety of mechanisms and locations. We propose a novel approach to finding shared datasets: using natural language processing (NLP) techniques to identify declarations of dataset sharing within the full text of primary research articles. Using regular expression patterns and machine learning algorithms on open access biomedical literature, our system was able to identify 61% of articles with shared datasets with 80% precision. A simpler version of our classifier achieved higher recall (86%), though lower precision (49%). We believe our results demonstrate the feasibility of this approach and hope to inspire further study of dataset retrieval techniques and policy evaluation.

Full text

***

My inspiration for this work was the idea of a Data Reuse Registry and associated research. As discussed, a DRR would benefit from automatic identification of data reuse in the biomedical literature. Unfortunately, automatic identification of data reuse is a tough place to start my NLP (natural language processing) journey because I haven’t found any large, pre-existing gold standards of data reuse to use for evaluating such a system (this list of GEO “third party” data reuse papers is a start).

Identifying data sharing is easier: there are available gold standards via database links, and authors tend to use more uniform language in describing sharing than reuse. Automatically detecting data sharing could be useful to my research in other ways as well, down the road, as I look towards further sharing policy evalutation.

This data sharing identification system used very simple NLP techniques. Hope to (and will probably need to) dig into some more complex approaches as I tackle data reuse identification.

If anyone knows of other resources that list specific instances of data reuse, I’d love to hear about them!

It continues to surprise me how hard [1, 2] it is to figure out the rules for publishing in most journals, particularly with respect to preprints. I’m finding the policies for publishing in conferences are just as ill-defined. This is a shame because it surely decreases sharing.

I’ve been putting my work up on Nature Precedings. I submitted a poster a few days before the ISMB conference. Nature Precedings asked me if I knew whether ISMB would permit this. I emailed to ask: they quickly responded that it was no problem. I’ve since asked PSB and ELPUB: again, no problem. I think that ELPUB did make state this during in the submission process, but in all three cases it would be great if they wrote this explictly on their websites so that people could feel free to post and discuss draft submissions… I’ll send a few emails to suggest it.

I recently asked AMIA. I was pointed to the contributor license agreement [below], which left me guessing. What does “publish” mean? For most publishers I’ve talked to it means “published in an official capacity” and they don’t count blogs or pre-print servers. Yet clearly the journals who disallow preprints disagree with this definition [quick list for Clinical Medicine, or Sherpa/RoMEO for an extensive list]. JAMIA, the official journal of AMIA, does not allow preprints. So I guess I should email AMIA again and ask for further clarification.

ETA:  I emailed again and quickly got a very clear response: “Preposting is fine so long as the other sites don’t formally publish the work.”   Great news, thanks AMIA.

I’d like to “ask for forgiveness rather than permission” but the potential down-side is too great in this case. Student publication in AMIA is highly-valued in my department.

I wonder if author copyright addenda can apply to conference submissions??? I don’t see why not. Will have to keep that in mind next time.

AMIA Annual Symposium 2008 license agreement: