Access Problem Solving (digprob)
Directory of Open Access Journals (DOAJ) Import to Vera
A. Downloading and processing the file from the DOAJ
1. In your web browser, go to http://www.doaj.org/doaj2csv.cgi. You'll get a dialog box titled "Downloading doaj" that will ask you what your browser should do with the file. Save it to your Vera Import folder.
The file from the DOAJ doesn't contain all the information we need for the Vera records, nor does it contain all the information in the correct place. Rich Wenger wrote a script that will fix these things for us. The script to make these changes is called ej_format.pl, and resides on Athena at:
/afs/athena.mit.edu/dept/libraries/staff/systems/scripts/openaccess2. In SecureFX, connect to Athena using your DOAJ Script profile. This will connect you to the path listed just above.
3. In the left hand pane, rename the file doaj2.csv (add a period between the 2 and the c).
4. Copy the file doaj2.csv from your local directory to the Athena directory.
5. In SecureCRT at an Athena prompt, type cd /afs/athena.mit.edu/dept/libraries/staff/systems/scripts/openaccess to get to that directory (that's cd, space, and then the path name)
6. Type pwd to make sure you're actually in the right place.
7. To run the script, type the following exactly as it is written below:
./ej_format.pl --infile=doaj2.csv --outfile=doaj-done.csv
8. After the script finishes running, return to SecureFX. Make sure to Refresh Views under the View menu, then FTP the finished file called "doaj-done.csv" back down to your computer into your Vera import folder.
B. Post script-running process
- Launch Excel, and open a new blank document.
- In the Data menu, choose "Import External Data" then "Import Data File...". Navigate to the file on your computer and then choose Open.
- When confronted with the Import Wizard, you'll have to make some choices. Your answers are
- Step 1: delimited (then click Next)
- Step 2: comma (uncheck Tab, then click Next)
- Step 3: general (then click Finish)
- Where do you want to put it? Existing worksheet.
- Now you can eyeball it to see if it looks like the data is in the correct columns.
- Save the document as an Excel file. (.xls). Go to File --> Save As... and
- Save In: your Vera import folder
- File name: doaj-done.xls
- Save as type: Microsoft Excel Workbook (*.xls)
- You now have a file called "doaj-done.xls." This is the file you'll import into Vera.
C. Deleting the set imported into Vera last time
Before you import the current set, you have to delete the old set.
1. In Vera, find the set of records you imported from DOAJ last time by doing a Find in the Mark Set field for the text "imported from DOAJ."
2. Once you're sure you have the correct set of records, go to Records --> Delete Found Records.
D. Importing the DOAJ file into Vera
- In Vera, go to File --> Import records
- Navigate to the file on your computer "doaj-done.xls" (if it asks, it's Sheet1)
- Check the box "Don't import first record (Contains field names)."
- Match the field names on the left (how the columns are labelled in done.xls) with the field names on the right (what they are called in Vera). If you choose View By: Matching Names, it should work.
- Make sure you have the arrow pointing from left to right (-->) between each of the field names above.
- Make sure you uncheck everything else, so that there is a little zero (kind of looks like a dot) for everything else (i.e., not a --> and not a <-->)
- Check to make sure your choices make sense by "scanning" the data.
- Choose "Add new records" radio button
- Click Import button
- Check the box that asks if you want to "perform auto-enter options." (You do want to perform auto-enter options; this is how the unique Vera ID is created, how the creation date gets in there, the Creator Name, etc.)
- Click OK.
- Note the number of records imported and email to Kim so it can be added to the History of records imported in the table at the bottom of the page. There is significant duplication between the DOAJ and J-STAGE journal lists. In order to avoid duplicating the open-access records that come in the J-STAGE load, do a find with "DOAJ" in the Mark set field, and "http://www.jstage" in the URL_Native field. Note the number of records returned. The delete the returned set with "Delete fournd records." Subtract the number of J-STAGE records deleted from the total original number of DOAJ records imported, and e-mail this figure to Kim so it can be added to the DOAJ import history.
E. Global Replaces for the Set
Mark Set
- Add the text "imported from DOAJ, <today's date>" to the Mark Set field of the first record in the set.
- Click in the field so that your cursor is in the field, then go to Records --> Replace...
- It will ask if you want to replace the contents of the Mark Set field to whatever you just typed in the [number of] records in the found set. You have to actually choose Replace by clicking on it (Enter will cancel the process.)
History of records loaded into Vera
Records should be loaded into Vera every two months.
Date 5/13/2004 11/12/2004 3/21/2005 5/20/2005 7/15/2005 9/22/2005 11/21/2005 12/2/2005 1/20/2006 1,922 (does not include the 85 records that are also part of JSTAGE and which we deleted from this DOAJ load) 6/1/2006 2,045 (does not include the 173 records that are also part of JSTAGE and which we deleted from this DOAJ load) 7/27/2006 2,229 (does not include the 92 records that are also part of JSTAGE and which we deleted from this DOAJ load) 9/29/2006 2,315 (does not include the 86 records that are also part of JSTAGE and which we deleted from this DOAJ load) 11/17/2006 2,381 (does not include the 86 records that are also part of JSTAGE and which we deleted from this DOAJ load) 6/1/2007 2,633 (does not include the 86 records that are also part of JSTAGE and which we deleted from this DOAJ load) 8/10/2007 2,718 (does not include the 85 records that are also part of JSTAGE and which we deleted from this DOAJ load) 10/6/2007 2,773 (does not include the 85 records that are also part of JSTAGE and which we deleted from this DOAJ load) 12/21/2007 2,946 (does not include the 83 records that are also part of JSTAGE and which we deleted from this DOAJ load) 2/15/2008 3,104 (does not include the 81 records that are also part of JSTAGE and which we deleted from this DOAJ load) 3/29/2008 3,222 (JSTAGE was down so couldn't get that file) 5/2/2008 3,258 (does not include the 82 records that are also part of JSTAGE and which we deleted from this DOAJ load) 11/6/2008 2/24/2009 3788 titles (3861 gross, minus 73 JSTAGE titles) 5/7/09 4045 (4124 gross, minus 71 JSTAGE titles) 10/9/09 4280 (4360 gross, minue 70 JSTAGE titles)