I have some portable weather stations that are using the CR10X datalogger, and we are updating to the CR1000 because we need some additional functionality.
First, a description of our weather stations. We are setting up portable stations similar to the RAWS systems, but ours are used for flight testing, often in remote locations where there is no other weather data available. Our systems are set up temporarily, monitored in real time (using RTMC), and then torn down at the end of the day with data parsed by SPLIT and saved for historical data.
We have the following sensors:
HMP45C (temp, rh)
CM3 (solar radiation)
CS105 (barometric pressure)
05103 wind sensors with 32500 electronic compass
We'll be updating the wind sensors soon, likely to RM Young 85000 series ultrasonic sensors.
We display measurements of wind speed/direction and solar radiation once every second. Every 60 seconds I display temp, RH, and pressure. Every five minutes I display gust data, prevailing wind direction, and standard deviation of wind direction for the past five minutes.
I have all this working on the CR10X and ported over to the CR1000 as well just fine, but I would like to make some refinements on the CR1000. List of things I want to change:
1. Add GPS for the following:
a. position data, only needs to be stored once per test setup.
b. time data, used to set station time to local time. This presents a couple issues (how does end user set UTC offset without editing .CR1 file? Is daylight saving time taken into consideration automatically, or need to be set by user?)
c. Use $GPRMC string to get Magnetic Variation, which would then allow true north to be displayed instead of magnetic north.
2. I'd like to change the five minute averages and standard deviation to be a rolling five minutes instead of 'chunk' data. I assume this is possible with the CR1000 but haven't started working on it yet.
Here's where I'm stuck. I have a test program working for the GPS data, using the samples in the GPS16X-HVS manual. There were some errors in the example that I fixed, but I am still having a problem with running the examples concurrently. It seems I can either store the GPS data in an array and then use the GPS command in the main program to set station time first example), or I can store the individual GPS strings in table format (second example) but if I use the GPS command in the main program, this breaks the tables and fills them with empty data.
I also noticed that there is a problem with the second example in that it is looking for the position of data in string, not for the comma delimiters. This means that if any data is absent, all of the subsequent pieces are incorrect. I'd rather not use this method as it appears to be unreliable.
So how do I write the arrays from the first example (page 8,9 in the manual) to a data table as in the second example (page 10,11)? I can't seem to get it to work.
I've answered my last question in the above post with regard to how to get the data out of the array in example one. I still need answers to the other questions, but now I've got a place to start.
This is the working GPS program (adapted from example 1, with the fixes).
'CR1000 Series Datalogger
'GPS Variables
Const LOCAL_TIME_OFFSET = -5 'Local Time offset relative to UTC time
Dim nmea_sentence(2) As String * 90
Public gps_data(15)
Alias gps_data(1) = latitude_deg 'Degrees latitude (+ = North; - = South)
Alias gps_data(2) = latitude_min 'Minutes latitude
Alias gps_data(3) = longitude_deg 'Degress longitude (+ = East; - = West)
Alias gps_data(4) = longitude_min 'Minutes longitude
Alias gps_data(5) = speed 'Speed
Alias gps_data(6) = course 'Course over ground
Alias gps_data(7) = mag_var 'Magnetic variation from true north (+ = East; - = West)
Alias gps_data(8) = fix_quality 'GPS fix quality: 0 = invalid, 1 = GPS, 2 = 'differential GPS, 6 = estimated
Alias gps_data(9) = nmbr_sat 'Number of satellites used for fix
Alias gps_data(10) = altitude 'Antenna altitude
Alias gps_data(11) = pps 'Elapsed ms since last pulse per second (PPS) from GPS
Alias gps_data(12) = dt_since_gprmc 'Time since last GPRMC string, normally less than '1 second
Alias gps_data(13) = gps_ready 'Counts from 0 to 10, 10 = ready
Alias gps_data(14) = max_clock_change 'Maximum value the clock was changed
Alias gps_data(15) = nmbr_clock_change 'Number of times the clock was changed
'Define Units to be used in data file header
Units latitude_deg = degrees
Units latitude_min = minutes
Units longitude_deg = degrees
Units longitude_min = minutes
Units speed = m/s
Units course = degrees
Units mag_var = degrees '(+ = East; - = West)
Units fix_quality = unitless
Units nmbr_sat = unitless
Units altitude = m
Units pps = ms
Units dt_since_gprmc = s
Units gps_ready = unitless
Units max_clock_change = ms
Units nmbr_clock_change = samples
DataTable (gpsdata,True,-1)
DataInterval (0,1,Sec,10)
Sample (1,latitude_deg,String)
Sample (1,latitude_min,String)
Sample (1,longitude_deg,String)
Sample (1,longitude_min,String)
Sample (1,speed,String)
Sample (1,course,String)
Sample (1,mag_var,String)
Sample (1,fix_quality,String)
Sample (1,nmbr_sat,String)
Sample (1,altitude,String)
Sample (1,pps,String)
Sample (1,dt_since_gprmc,String)
Sample (1,gps_ready,String)
Sample (1,max_clock_change,String)
Sample (1,nmbr_clock_change,String)
EndTable
'Main Program
BeginProg
Scan (1,Sec,0,0)
GPS (gps_data(),Com4,LOCAL_TIME_OFFSET*3600,0,nmea_sentence(1))
CallTable gpsdata
NextScan
EndProg* Last updated by: MichaelM on 8/3/2010 @ 9:59 AM *
position data, only needs to be stored once per test setup.
This will read the GPS once and store the Lat and Lon for you. If you want to update, just have something make GPSread false. I used Port4 to turn the GPS on and off when not in use, it draws ~70 mA if memory serves me correctly.
'Routine to Read GPS
If GPSread = False Then
If gps_ready <> 10 Then
PortSet (4,0 ) 'Turn on GPS
GPS (latitude_a,Com3,UTC_OFFSET,0,nmea_sentence(1)) ' Read GPS
ElseIf gps_ready = 10 Then
Lat = latitude_a + (latitude_b / 60) 'Calculate Decimal Latitude
Lon = longitude_a + (longitude_b / 60) 'Calculate Decimal Longitude
Alt = altitude
PortSet (4,1 ) 'Turn off GPS
GPSread = True 'Mark GPS as read
EndIf
EndIf
Change this:
Const LOCAL_TIME_OFFSET = -5
To This:
ConstTable
Const LOCAL_TIME_OFFSET = -5
EndConstTable
It will allow the customer to change the offset with a keyboard display, BUT BE CAREFUL, it will recompile the program so make sure to save data if required before you change.
Magnetic Variation:
Alias gps_data(7) = mag_var
I think that may answer the questions.....
Good Luck!
* Last updated by: IslandMan on 8/4/2010 @ 4:31 AM *
Islandman, thank you. The routine for the GPS appears to work perfectly for my needs.
I can make the ConstTable thing work too, but it won't be pretty. I'll probably use AutoHotKey to write a script to see if I can emulate the terminal with minimal user interface. I think it can be done.
I've already got the Magnetic Variation working shortly after I posted my original question and response.
I'll keep this thread updated as I come across other issues/solutions.
If you explain a little more about how you want to execute the 5 minute running average I may be able to pull some code for that as well.
We are looking at 'gust' data (maximum wind speed kn knots) and prevailing wind direction with the standard deviation over five minute intervals.
With the CR10X, we were limited to chunks. In other words, our five minute averages were, for example, at 11:00, 11:05, 11:10, etc.
What I would like is to know the prevailing wind direction and maximum wind speed over the last five minutes at any given time.
So, once the station is powered up, it would take five minutes to compute the first averages (term used loosely to describe both the gust and the prevailing wind direction with standard deviation), then after that it would constantly compute this data, updated once per second, going back the last five minutes. Make sense?
I appreciate your help in this regard.
Michael,
I will let Dave answer your running average question (since he's probably got it half written by now...), but also take a look at the custom menu capability in the datalogger, and in particular the MenuRecompile instruction. It will let you add a pick list for a Constant Table item, which may allow for a little easier UI for the end user.
Dana W.
Michael,
Give this a try. You have to add the Wind sensor measurements.
'CR1000 Series Datalogger
'Declare Public Variables
Public Wspd
Public WspdGust(300)
Public GustCount
Public Wdir
Public SinWdir
Public CosWdir
Public AvgSinWdir
Public AvgCosWdir
Public e
Public AvgWspd
Public AvgWdir
Public Sigma
Public Valid
Public MaxGust(2)
Alias MaxGust(1) = Gust
'define all angles in degrees not radians
AngleDegrees
'Main Program
BeginProg
'Init Variables
Valid = 0
GustCount = 1
Scan (1,Sec,0,0)
' Do not output Averages or Sigma until all 300 samples are acquired
If Valid < 300 Then
AvgWspd = 0
AvgWdir = 0
Sigma =0
Gust = 0
Valid = Valid + 1
EndIf
'Measure Wind Speed an Direction Sensor
'Store Wind Speed readings in array for '"Gust" calculation
If Gustcount < 300 Then
WspdGust(Gustcount) = Wspd
Gustcount = Gustcount + 1
Else
Gustcount = 1
EndIf
SinWdir = SIN(Wdir) ' 10M WD Sine
CosWdir = COS(Wdir) ' 10M WD Cosine
AvgRun (AvgWspd,1,Wspd,300) 'Average Wind Speed
AvgRun (AvgSinWdir,1,SinWdir,300) 'Average Wdir Sine
AvgRun (AvgCosWdir,1,CosWdir,300) 'Average Wdir Cosine
'Calculate 5 Minute Running averages for Wind Direction
AvgWdir = ATN(AvgSinWdir/AvgCosWdir)
If AvgCosWdir < 0 Then
AvgWdir = 180 + AvgWdir
ElseIf AvgSinWdir < 0 Then
AvgWdir = 360 + AvgWdir
EndIf
'Calculate 5 Minute Running Sigmas for Wind Direction (yamartino method)
e = SQR (1-((AvgSinWdir^2)+(AvgCosWdir^2)))
Sigma = ASIN (e*(1+(0.1547*(e^3))))
'find Max Wind Speed (GUST) over 300 samples
MaxSpa (Gust,300,WspdGust())
NextScan
EndProg
Dave,
Thanks again for sharing your expertise!
I'm bumping an old thread with a new (related) question.
After some testing in the field and some toying with data, I'd like to read the GPS more than once, something on the order of once every 15 minutes or so. The trick is, I don't want to start recording any data for any of the other sensors unless and until there is an initial GPS lock, then start recording from there.
Essentially (leaving out the non-relevant parts):
'Declares
Public GPSread = False
Public GPSInit = False
'Tables
DataTable (gpsdata,True,-1)
'No DataInterval since calling table conditionally
Sample (1,lat,string)
Sample (1,lon,string)
Sample (etc., etc. etc.)
EndTable
DataTable (1_Sec_Data,True,-1)
DataInterval (0,1,Sec,10)
Sample
Minimum
etc.
EndTable
DataTable (1_Min_Data,True,-1)
DataInterval (0,1,Min,10)
Sample
Minimum
etc.
EndTable
'Main Program
BeginProg
Scan (500,mSec,10,0)
'Routine to Read GPS
If GPSread = False Then
If gps_ready <> 10 Then
PortSet (3,0) 'Turn on GPS
GPS (gps_data(),Com4,UTC_OFFSET,0,nmea_sentence(1)) ' Read GPS
ElseIf gps_ready = 10 Then
Lat = latitude_deg + (latitude_min / 60) 'Calculate Decimal Latitude
Lon = longitude_deg + (longitude_min / 60) 'Calculate Decimal Longitude
Alt = altitude
PortSet (3,1) 'Turn off GPS
CallTable gpsdata
GPSread = True 'Mark GPS as read
GPSInit = True 'Start recording all other sensors
EndIf
EndIf
'Measure One Second data here
'...
'Store Data in 1_Sec_Data table
If GPSInit = True Then CallTable 1_Sec_Data
'Measure One Minute data here
'...
'Store Data in 1_Min_Data table
If GPSInit = True Then CallTable 1_Min_Data
'Here's where I run into problems
If IfTime (14,15,Min) Then GPSread = False
NextScan
EndProg
The GPS data in the Public table (where it is initially stored) is never updated after the first run, so all records in the DataTable gpsdata are copies of the original reading.
I'm sure I'm missing something simple here, but I can't figure where I'm going wrong. Everything else in my code works just as I expect it to, only the subsequent GPS reads are currently not working.
* Last updated by: MichaelM on 2/4/2011 @ 12:17 PM *
Hi Mike,
When you do this:
'Tables
DataTable (gpsdata,True,-1)
'No DataInterval since calling table conditionally
Sample (1,lat,string)
Sample (1,lon,string)
Sample (etc., etc. etc.)
EndTable
You want to set this:
DataTable (gpsdata,True,-1)
to something like this:
DataTable (gpsdata,True,1000)
A data table without a data interval and the -1 will auto allocate enough memory to store, in your case, an output every 1/2 second which is a ton.
Here is your program, revised to work, hopefully:
'Declares
Public GPSread As Boolean
Public GPSInit As Boolean
'Tables
DataTable (gpsdata,True,1000)
'No DataInterval since calling table conditionally
Sample (1,lat,String)
Sample (1,lon,String)
Sample (etc., etc. etc.)
EndTable
DataTable (1_Sec_Data,GPSInit,-1)
DataInterval (0,1,Sec,10)
Sample
Minimum
etc.
EndTable
DataTable (1_Min_Data,GPSInit,-1)
DataInterval (0,1,Min,10)
Sample
Minimum
etc.
EndTable
'Main Program
BeginProg
GPSread = False
GPSInit = False
Scan (500,mSec,10,0)
'Routine to Read GPS
If GPSread = False Then
If gps_ready <> 10 Then
PortSet (3,0) 'Turn on GPS
GPS (gps_data(),Com4,UTC_OFFSET,0,nmea_sentence(1)) ' Read GPS
ElseIf gps_ready = 10 Then
Lat = latitude_deg + (latitude_min / 60) 'Calculate Decimal Latitude
Lon = longitude_deg + (longitude_min / 60) 'Calculate Decimal Longitude
Alt = altitude
PortSet (3,1) 'Turn off GPS
CallTable gpsdata
GPSread = True 'Mark GPS as read
GPSInit = True 'Start recording all other sensors
EndIf
EndIf
'Measure One Second data here
'...
'Store Data in 1_Sec_Data table
CallTable 1_Sec_Data
'Measure One Minute data here
'...
'Store Data in 1_Min_Data table
CallTable 1_Min_Data
'Here's where I run into problems
'If Time = 15 minutes Read GPS
If IfTime (0,15,Min) Then
gps_ready = 0 ' Reset GPS Ready
GPSread = False
GPSInit = False ' Stops the data recording
EndIf
NextScan
EndProg
Good Luck!!!
Thanks Dave.
I've rewritten some of my code to address much of what you suggested. I've tried resetting gps_ready=0 and GPSread = False (but not GPSInit since I want the rest of my data to continue to update on schedule). I'm also reading the C3 to ensure that it does indeed turn off the GPS, and it does.
What's happening now is that after the first scan things behave as expected until I set the GPSread flag False again.
Once GPSread goes false, one second later C3 goes low, gps_ready = 10 (instead of counting like it should) and GPSread hits True, and the Table is updated with the initially stored Public table data and C3 goes high again.
In fact, all of the gps data in the Public table never changes after the initial write to the gpsdata Table.
* Last updated by: MichaelM on 2/4/2011 @ 1:58 PM *
First regarding the program problem. You need to reset gps_ready to 0 after you get a good reading, otherwise it will never call GPS again.
A couple of points about this discussion
1) If you have the power available, let the GPS run all the time. Reason is that using the PPS signal and the GPS data the logger will sync into and run off the GPS clock as its own master clock giving you accuracy of time down to 10 ms or even better with recent loggers.
IF you do not do this it will revert to its own clock when it loses the GPS clock and then make a sudden adjustment when the GPS comes on line again. Adjusting the clock like this too often can cause the skipped scans and records, although you can limit how often this happens by setting the allowable error to not too tight a limit.
2) The WMO definition of a Wind Gust is the max of a 3 second average, so you should do a 3 second rolling average of windspeed and take the max of that over a longer period.
Sorry emails partly crossed. The problem with the latest issue you report is probably because the GPS instruction does not operate quite as it seems. Once called it acutally kicks of a background process that runs all the time trying to use the PPS signal as its master clock. When you call the instruction again it only updates the GPS string and other variables with the last good values captured. I suspect it need one extra execution to get the gps_ready parameter to give the latest value as I recall once the times are synched the values only get updated close to the top of the scan.
In your case it would probably be best to call the GPS instruction every scan, not conditionally, but turning the power on and off and monitoring the gps_ready counter to trigger the data storage much as you have now.
It was my understanding you only wanted to collect the 1 min and 1 sec tables when you had a good GPS signal, hence the
DataTable (1_Sec_Data,GPSInit,-1) in the data tables. The tables are called on every scan but only written when GPSinit is True.
I looked at my original GPS code I sent way back when and should have had more leeway in the amount of time it was allowed to be off before resetting the clock. The sample I gave you had zero and the manual recommends a minimum of 20ms.
Unfortunately, I don't have a GPS sensor here at the office so I couldn't test the code :-)
* Last updated by: IslandMan on 2/4/2011 @ 7:29 PM *
Dave,
When set up these weather stations, it is for a few hours during a flight test, at most it will be just prior to dawn to just after sunset. We only test during fair weather and only during daytime unless the test specifically calls for night time testing (which does occasionally happen).
So my plan was to get a GPS lock to set the initial datalogger time, sync the CR1000 to that, and then let the datalogger free run afterward. The datalogger shouldn't drift enough during a few hours to cause any issues. This is how I've been using it since I originally started this thread last year.
What we've noticed is the GPS position wanders a fair amount and can easily reach its published 15 meter error. However, doing some testing in the lab I discovered that if I average the GPS coordinates over time I can get within about 3m accuracy. Not a big deal, really, but if we can get closer to the actual position then that would be nice.
At the moment I'm taking aps's advice to leave the gps on and read it continuously. I've changed the code to use the continuescan function if gps_ready <> 10. This gets me the initial fix prior to scanning any other sensors. I'm also reading the gps every scan and storing every 5 seconds using average. I'm finding this to be relatively stable and increasingly accurate in lat, lon, and altitude, but I might end up drawing more power than I'd like. If power becomes an issue I'll just go back to the original scheme of scan once and turn off, and we'll live with the inaccuracies of the position fix.
I appreciate you checking back in on this thread. Your help has been greatly appreciated!
Mike,
Are you using the GPS to sync the time in all of the loggers for the test? If there is communication between the loggers you might want to set one logger and use ClockReport to set all the others. Just a thought.
Sometimes it's good to know ahead of time what you are trying to do with a program to best offer solutions. I learn lots reading this board and in turn try to help out where I think I can :-)
Usually we just deploy one station during testing per location, and just when we go to remote locations where real-time weather is otherwise not available.
I'm still tweaking the program a bit but at the moment it's looking pretty good. I plan to field check them in the coming weeks when the weather outside gets a bit better.