Was this page helpful?

Rawarch plugin

From $1

    This plugin serves to record raw data along with some of the parameters describing it. It is intended
    to allow archiving of data from any of the sources supported by radR, without making any of the data
    reduction assumptions involved in recording blipmovies. Raw archive files will generally be much larger than blipmovie files. You can write a small R program to convert your own binary data into a raw archive; see here for instructions.

    Format of a raw archive

    The rawarch plugin uses radR's biglist and bigframe packages to save data as a big R list in a file.
    In what follows, we assume the variable bl refers to a biglist object created in the following way:

    > library("bigframe", lib.loc="/path/to/radR")  ## load the bigframe package

    > library("biglist", lib.loc="/path/to/radR")   ## load the biglist package

    > bl <- biglist("mybigfile.raw")                ## open an existing rawarch file


    After loading them, you can use  ?bigframe and  ?biglist to learn more about these packages.
    The file mybigfile.raw contains biglist data, and will have an accompanying index file called mybigfile.raw.i
    This index file is actually a bigframe.

    1. The header (item 1)

    The first item in the biglist is a table of contents for the archive, because the raw archive
    can contain multiple runs (i.e. sequences of consecutive scans):

    > bl[[1]]
    [1] 711 261  29  37  32  69
    [1] 1141674091 1141677352 1141678442 1141678651 1141679353 1142455575
    [1] 1141675912 1141678014 1141678516 1141678742 1141679432 1142455756

    The table of contents is a list with three elements:

    • num.scans:  (numeric or integer) the number of scans in each run (the length of this vector is the number of runs)
    • start.time:  (integer) timestamps of the first scan in each run
    • end.time:  (integer) timestamps of the last scan in each run

    Timestamps should be relative to universal time, and can be converted to date times as so:

    > structure(bl[[1]]$start.time, class="POSIXct")

    [1] "2006-03-06 14:41:31 EST" "2006-03-06 15:35:52 EST"
    [3] "2006-03-06 15:54:02 EST" "2006-03-06 15:57:31 EST"
    [5] "2006-03-06 16:09:13 EST" "2006-03-15 15:46:15 EST"

    Note that the total number of scans in this raw archive is given by:

    > sum(bl[[1]]$num.scans)
    [1] 1139

    2. The scan parameters (items 2, 4, 6, ...)

    The even numbered items in the biglist are the parameters for scans 1, 2, 3, and so on. Scans parameters are stored in the order dictated by the table of contents.  For the file in the example, the parameter sets for the 711 scans from the first run are in biglist items 2, 4, 6, ..., 1422; these are followed by parameter sets for the 261 scans from the second run in biglist items 1424, 1426, 1428, ..., 1944; and so on.

    For example, the parameters for the first scan are:

    > bl[[2]]
    [1] 1024              ## the number of pulses in the first scan

    $samples.per.pulse    ## the number of samples per pulse (i.e. range cells)
    [1] 2048

    $bits.per.sample      ## the bit resolution of each sample
    [1] 12

    [1] "2006-03-06 14:41:31 EST"    ## this is an integer of class "POSIXct"

    $time.offset                     ## for more precise timing, this is the
    [1] 0.2430000                    ## fractional seconds after timestamp of the
                                     ## start of the scan

    [1] 2609                         ## duration of the scan, in milliseconds

    $sample.dist                     ## distance covered by one sample, in metres
    [1] 4.996542                     ## (i.e. the size of a "range cell")

    $first.sample.dist               ## distance from the radar of the first sample
    [1] 0                            ## (i.e. range of the first "range cell")

    $bearing                         ## degrees clockwise from north of the first pulse
    [1] 0

    $orientation                     ## radar rotation direction: +1 = clockwise, -1=counterclockwise
    [1] 1

    $latitude                        ## geographic location of the radar antenna(optional)
    [1] 45.08933

    [1] -64.36908

    3. The raw scan data (items 3, 5, 7, ...)

    The odd numbered items in the biglist (except for item 1) are string vectors with single elements containing the raw scan data in binary form.  Scans are stored in the order dictated by the table of contents.  For the file in the example, the 711 scans from the first run are in biglist items 3, 5, 7, ..., 1423; these are followed by the 261 scans from the second run in biglist items 1425, 1427, 1429, ..., 1945; and so on.

    Unlike in C, strings in R can have embedded NULL (0x00) characters .  The only reason we don't use an R raw vector is that (surprisingly) R handles raw vectors much less efficiently than character vectors when reading/writing them from/to files.

    Samples are packed as tightly as possible into the R string.  Samples occupying more than one byte have their data stored in order from least significant to most significant (i.e. little-endian), as is standard for i386-family computers.


    > class(bl[[3]])
    [1] "character"

    > length(bl[[3]])
    [1] 1

    > nchar(bl[[3]])
    [1] 3145728   ## 12-bit data is packed so 3 bytes holds two samples: 3145728 = 2048 * 1024 * 3 / 2

    > charToRaw(substr(bl[[3]],1,16))
    [1] 67 ba 8d 4b cc 09 4b ab 6e 1f 9a 88 46 99 b5 f7       ## the first two 12-bit values are:  0x0a67, 0x08db

    The the nibble-packing order for 12-bit data is as follows:

         input:     byte0    byte1    byte2
         nibble:    A   B    C   D    E   F
                    lo hi    lo hi    lo hi    

         output:    short0           short1
                    A   B   C   0    D   E   F   0
                    lo         hi    lo         hi

    i.e. two 12 bit words: 0x0CBA, 0x0FED  <--> 0xAB, 0xCD, 0xEF (3 bytes)

    Was this page helpful?
    Tags: (Edit tags)
    • No tags
    Comments (0)

    Powered by MindTouch Core