Manual Upload

##Manual Upload Instructions### These instructions assume that you will ssh into the worldwater server and use the code that resides there, or that you will have the knowledge required to install the necessary python modules.

###File Format To run manually, data_transfer.py must be executed with the -xls option and a file in .xls format. 5G0E3559-sample.xls is a sample xls file to show correct formatting. The files will have two tabs; the scripts do not upload the data from the "Unprocessed Data" tab. data_transfer.py uses functionality from decagon.py to parse the xls files.

The filenames have spaces when downloaded. There is a handy little script called name_fixer.py that can be used to change the names of a directory of files to remove all spaces like this:

python name_fixer.py [name_of_directory]

If you have trouble with it, run with the -h option to see help output:

python name_fixer.py -h

###Running the Script Before running manual upload, use MySQL Workbench to query the most recent upload date for the datalogger. The datalogger name is in each filename from the loggers. For example, if the filename is "5G0E3559 27Mar15-1046.xls", the datalogger name is "5G0E3559". Use the lookup table "01-LookupTable.xlsx" to find the site codes for that datalogger, in the Site column of the table (Sheet 1). Use those codes to execute the following SQL statement in MySQLWorkbench (you can find Workbench help here):

SELECT LocalDateTime FROM rush_valley.datavalues AS dv  
JOIN rush_valley.sites as s 
ON s.SiteID = dv.SiteID 
WHERE s.SiteCode IN ("Ru1BMP5", "Ru1BMP30", "Ru1BMPA", "Ru1BMNU")
ORDER BY LocalDateTime DESC LIMIT 1;

Replace "Ru1BMP5" and the other SiteCodes with the codes from the lookup table. The result of the SQL query will be a timestamp of the most recent upload from that logger to the database. Values older than that should already be present, so to avoid duplicates we include the timestamp. Use that timestamp (right-click it in Workbench, click "Open Value in Viewer", go to "Text" tab, copy and paste) as the -lt argument for data_transfer.py. The command to run the upload should look like this:

python data_transfer.py -lt '2015-06-15 00:00:00' -xls manual_files/rush150326/5G0E3559_27Mar15-1046.xls

Don't forget to enclose the timestamp in quotes and make sure the full path to the data file is present.

Use the -h option to view help and other options (-v prints out information for debugging, -nu doesn't upload data to database)

python data_transfer.py -h