[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: RE: Data logging data base?
- From: Philippe Lhoste <PhiLho@...>
- Date: Fri, 20 Jul 2001 10:28:45 +0200 (MEST)
John Passaniti wrote:
> > Because the data is sampled every three seconds for
> > several days, this data base could get large fast.
> > However I'm more interested in speed than disk space.
> > Tho I don't want have to take up all of the computer
> > memory with the data file.
> Use the filesystem as the database. That is:
> > YEAR:MONTH:DAY:SECONDS_FROM_MIDNIGHT as the key, then
> > a channel number in array format, to get the values.
> The directory structure might look like:
> Within the final directory, either have each sample as a separate file, or
> bundle them into a single file. I would probably make each a
> separate file.
I am not a specialist of Unix, but where I work, we use a similar scheme
(sub-sub-directories, generated by hash, not by date) to store messages of users
for a web mail program.
It is nice and easy, but for very large databases, it seems there is a
problem of a limit of the number of available i-nodes.
Again, I don't know the exact figures, but you may be aware of this problem,
and perhaps aggregate data at a higher level (by hour or by day).
Of course, I assume here, as you do, that is on a Unix system. Windows or
other systems may have problems of their own.
[...]> So to get back the table of sample values for a particular time:
> x = dofile("2001/7/19/2/6/9876543210")
Note: this works even on WinNT, ie. you can specify dofile("/2001/7/20/foo")
(current disk) or dofile("F:/2001/7/20/foo") (other disk) instead of
dofile([[F:\2001\7\20\foo]]). Great for compatiblity; and simplicity: the use of
backslashes forces to use the [[ ]] delimiters.
> > One of the other requirements is that I be able to
> > delete the data, on a day/date range bases.
> > Just put in NIL's?
> Want to delete all data for July 19th, 2001?
> rm -rf 2001/7/19
Again, assuming he is running on Unix... But it is easy to adapt to another
lhf: Thank you for the pointer, I will have more precise knowledge now :-)
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
Sent through GMX FreeMail - http://www.gmx.net