I've got a CR3000 with a 2Gb data card that's continually logging datd from sensors. The data card is setup as a 'ring memory', so that if the card is full, old data will automatically be over-written (we have good reasons for doing this, which I won't go into here).
Although our first few data retrievals worked perfectly, once the card was full and old data was overwritten, we encountered a problem with CardConvert. It seems the CardConvert software does not truly support ring memories, because if the timestamp of the data at the start of the file is later than the timestamp of the data onwards (i.e. earlier data has been overwritten as it's a ring memory) it fails to extract ANY of the earlier data.
I can see that the data IS in the file, via the viewer within CardConvert, but the data can't be extracted.
I have already reported this to Campbell Scientic, but the only solution they can offer is to manually split up the file so that the dates always increment. This is not a trivial task since the files are very large.
I really think this is a bug in CardConvert that should be fixed, and it seems to me it would be quite easy to fix in the software. Sure if the software/hardware supports ring memeory, the conversion software should allow this to be extracted?
Any feedback or alternatives that might help me get around this issue?
Thanks a lot!
Can you tell us which version of the Cardconvert you are running please and how you loaded it on the PC you are using (was it part of a Loggernet install). I ask as the Viewer should be using the same decoding DLLs as does Cardconvert.
If you need to get running quickly you have other options to do file conversions. You could try to use our Split program which is built into Loggernet, but there is a big learning curve to use that. Alternatively is you have Loggernet you should find an older program called TOB32 in the programs files directory of Loggernet which is designed to be used in batch files. There is also another program called FileFormat convert. Both are described in section B.4 of the Loggernet manual.
This from our engineering team:
If this is a TOB3 file (only format that rings), CardConvert should start with the oldest data in the ring memory and convert the whole file. An exception is if the ‘Convert Only New Data’ option is selected. In this case CardConvert only converts the new data. If CardConvert cannot determine what is new data (file has rung around etc.), it will convert the whole file starting with the oldest.
So it sounds like there might be a change needed to the processing options for the file.
If this is not the case, then contact your local CSI representative so we can get a copy of a file that exhibits the behavior.
Dana
Thank you for the quick responses!
aps: I'm using Cardconvert 1.7 (build 1.7.0.3). This was initially installed as part of a Loggenet install, but since I've experienced these issue a few months ago, just the CardConvert was upgraded to the version mentioned above via instructions of CS USA. Unfortunately this did not resolve it.
Dana: The file that I'm trying to extract the data from is a 2Gb .dat file, which is copied straight from the memory card (using a card reader on a laptop). The works perfectly as I've been using for all our data so far.
The "Convert only new data" is not selected when I read my data, so that is not the issue.
What the CardConvert produces is a single 4.5Gb, even though I've got the setting so that it's supposed to produce 1h files of data, which has worked perfectly for earlier files.
As I mentioned before, when I look at the data with the "View Files" within Cardconvert, I can a tiny bit of recent data right at the start of the file (less than an hour's worth), followed by old data that increments to to the end of the file. So clearly the new data has only just started to be overwritten.
I have also tried to repair the file, in case there any issues, but the repaired file produces the same result.
If the Viewer is showing the data out of order then this would appear to be either a bug in the code (which as I said is common to the viewer and Card Convert) or a corruption in the file. The guys in the US are best placed to look at this, but in preparation can I ask if you are able to ZIP the 2 GB file and put it on an ftp site somewhere so we can pick it up to check this out. Please send the access details to Dana by email (her email takes the form of firstname at campbellsci.com)
Meanwhile you could try out the TOB32 program I mentioned as that uses separate code for the conversion.
Thank you aps.
Where do I upload a 2Gb file?
Thank you for the TOB32 program suggestion. I did try it, and it does convert the file, but it does not offer the option of splitting up the file (I need to split it up in 1h chunks of data), which is what I need to do. It created a single 4.6Gb file.
We will contact you later with an option of where to put the file. (Zipping it first may reduce the size a little.)
Meanwhile if the TOB32 has decoded the data correctly into TOA5 format (or another CS format) you should be able to use the file format conversion program to split the data up as it includes an option to "bale" the files based upon time, e.g. to create hourly files.
Sorry about having to go through these steps as this should work with CardConvert whether the memory has rung around or not.
Hello Alex,
Please drop me an email so I can provide you with user name, password, and FTP site so you can upload the files.
Thanks, Dana
dana at campbellsci dot com
Would you inform us which type with the Cardconvert that you are running i highly recommend you and how you actually loaded it for the PC that you are employing (was its role of an Loggernet install). My partner and i ask as being the Viewer really should be while using the same breaking DLLs along with Cardconvert.
This is a following up after working with Alexv.
It appears that the data on the card was corrupt and an older operating system was in use. To prevent corrupt data make sure to use the latest OS. Support was added to handle larger card sizes along with some other fixes. Also be sure to follow the proper procedures for removing a card which includes using the Eject button first. The latest OS's can be found at www.campbellsci.com under Support | Downloads. If you have collected files that are not working properly, we have some steps that may be able to get you some useable data by using Card Convert and File Format Convert.