JCR’s Data Policy in Practice

What is JCR’s take on research integrity and data transparency? The journal’s policies on the subject can be found here. A key policy that was introduced under the previous editorial team is the data policy, which came into effect on 1 October 2020. We thought we’d explain how this policy works in practice and mention some other initiatives we will be putting in place in line with this policy to further JCR’s commitment to research integrity and transparency.

First things first – Who sets JCR’s policies?

You might be wondering where JCR’s policies come from, including the data policy. At JCR, policies are developed by the JCR Policy Board, not the editorial team. As editors, our responsibility is to manage the day-to-day implementation of the policies set by the Board. Given that each editorial team’s term is limited to three years, the Board provides necessary consistency over a longer period of time.

What do authors need to do?

All relevant information can be found on the JCR website and – we hope – it is sufficiently straightforward. For data, here are the main things authors need to do:

• Provide a data collection paragraph that gives full details of the data collection and who was involved and when. This is to be provided for each submission. All authors should sign off on this and make sure it is accurate.
• Provide in the manuscript (and, if necessary, in the web appendices) full details of the methods used to conduct the research. This includes things like data collection instruments, complete details of samples (including details on exclusions), complete details on procedures, and more – see the website for a list of everything. The more details authors can provide, the better. Transparency is the key.
• Provide in the manuscript (and, if necessary, in the web appendices) full details of how data were prepared and analyzed. As with methods details, this is all about transparency. If relevant, it is also a good idea to provide code for statistical analyses (particularly if they are more complicated or non-standard).
• Provide access to the data upon submission. More on this next.

Tell me more about the “data upon submission” requirement!

We require all manuscripts using data to provide, as part of the submission process, a URL that links to the data and all appropriate, relevant associated supporting materials (e.g., files explaining what is in the dataset, survey instruments, Qualtrics QSF files). This must be done in each round of the review process. One thing to keep in mind is that the review process is double-masked, so authors should be careful to make sure there’s no identifying information in the data files, survey/instrument files, or on the linked-to repository.

Does JCR have a preferred repository? 

No. To make this policy as easy on authors as possible, we let them decide where to put data and associated files. We have seen some authors link to a shared Dropbox or Google Drive folder, although most authors are using repositories such as OSF and ResearchBox (more repositories can be found here). We encourage the use of secure and established repositories instead of personal file-sharing services. We also suggest authors check with their universities as some universities might have institutional preferences on this, or even specific requirements.

What happens with the linked-to data upon submission?

The Editor and AE will see the URL on the manuscript information page in the journal’s editorial management system. When a manuscript comes in and before it is sent out for review the Managing Editor first checks that a link has been provided and that it works. If not, authors will be contacted and asked to fix this before anything else can happen. The next step is that the Editor assigned to the manuscript conducts some initial checks, including looking at the files in the repository to see if they appear to be the right types of files. For example, if a manuscript has four experiments we expect to see, at a minimum, four dataset files probably as Excel or CSV files. And yes, in case you are wondering, we do open the files! If something seems to be missing or doesn’t match our expectations, we will go back to the authors and ask them to check that they have uploaded everything correctly. If something needs fixing/updating, we let the authors do that before any further action in the review process.

Will the Editor and/or the review team re-run analyses?

The data policy as set by the JCR Policy Board does not specifically require this, but it can be done if deemed necessary by any member of the review team. Although only the Editor and AE can directly access the files during the review process, if a reviewer raises a question about the data that can only be addressed by them taking a look at the data we can – and will – provide them with the necessary access.

What about making all data publicly available?

At the moment, JCR does not require this for accepted manuscripts. Will this change in the future? Possibly, but for now we are seeing how the recently introduced policy on data upon submission works in practice. So far, authors have been great at following the policy, and we are encouraged by that. Of course, if authors wish to make their data publicly available (and are legally allowed to do so) then they are free to do so, which we think is a very good thing.

What if my data cannot be made available e.g., for legal or privacy reasons?

Sometimes this happens, for example where a dataset was provided by a company and is commercially sensitive, covered under a legal agreement such as a non-disclosure agreement (NDA), and therefore cannot be shared outside the author team. It can also happen in cases where providing data would violate participants’ rights to privacy (e.g., interview data). In these kinds of situations, it is worth talking to your university, as often making data available to outside entities such as journals is allowed for scientific review purposes. Remember, providing a link to a secure data repository that only can be seen by the Editor, AE, and Managing Editor is not making data public. In any case, where exceptions to the data policy may be required, it is best to ask us about it before submitting (send an email to jcr@ejcr.org).

How long should I keep my data after my manuscript is published in JCR?

In short, at a minimum seven years after the date of publication. The data maintenance policy for JCR is available on our website.

Is there anything else you are planning to do around data integrity and transparency?

Yes we are! We will provide more details on this soon, but later this year we will introduce a system of “data spot checks” for manuscripts that are at more advanced stages of the review process. In short, we will randomly sample advanced-stage manuscripts that report data-dependent results and submit them to a thorough set of empirical checks. This will be coordinated by the Editor and AE, who will look closely at all the provided files and materials, run some of the analyses, and generally make sure that everything seems to be in order. One thing we already do is put manuscripts through Statcheck, which can catch mistakes or errors in statistical reporting for commonly used statistical tests and reporting formats. We recommend that authors run Statcheck on their manuscripts prior to submission.