Processing Raw Data for Blips on Tracks

Inputs Required

• a blipmovie BM
• a .csv file of tracks output by the tracker plugin
• a function to process data from each blip.  This function must be assigned to the global variable blipfun

Running (as a script)

• start radR
• in the radR plot window, right click and choose "Source an R script..." | scripts/process_track_blips.R
• you will be prompted for the name of the track file
• you will then be prompted for the name of the blipmovie file
• if you haven't defined a function called "blipfun", then a default will be run. 
  This default prints a table with columns timestamp, trackNo, numPixels, area into a file called "TRACKFILE_blips.csv",
  if your tracks file is called "TRACKFILE.csv"

To see progress as the script runs, switch to the R terminal window.

Running (as a function call)

• start radR
• in the console, do

source("utils/processTrackBlips.R")

        which defines the function processTrackBlips.R

• define your function blipfun
• in the console, do

processTrackBlips("/path/to/blipmovie.bm", "/path/to/tracks.csv", blipfun)

Blip Processing Functions

The blip processing function is called for each blip in each track.  The function should be defined like so:

blipfun = function(sp, v, t, tn) {
## do something interesting
}

The parameters to the function (i.e. what the script supplies to your code) are:

• sp: the n x 2 matrix of sample / pulse coordinates for all "pixels" in a blip.  e.g. sp[5,1] is the sample index of the 5th pixel in the blip, and sp[10,2] is the pulse index of the 10th pixel in the blip.  The "sample" axis runs from the radar outward at constant angle.  The "pulse" axis runs clockwise around the radar at constant range.
  The matrix has attribute "index", which is a 2 element integer vector:
    attr(sp, "index")[1] : the index of the patch within all patches from this scan
    attr(sp, "index")[2] : if this patch is a blip, the index of the blip within all blips from this scan, otherwise 0.
  So you can get blip summary statistics as RSS$patches[attr(sp, "index")[1],]
• v: the raw sample value for each pixel in sp.  e.g. v[5] is the raw sample value for the 5th pixel, whose sample/pulse coordinates are sp[5, 1:2]
• t: the timestamp for the blip centroid, as appears for this blip in the tracks.csv file
• tn: the track.no field from the tracks.csv file

For example, the default blipfun in process_track_blips.R is defined like this:

#### print number of samples and area for each blip in a track

  blipfun = function(sp, v, t, tn) {

#### e.g. this writes the timestamp, track number, number of samples
#### (which is the number of rows in the matrix sp) and the blip area
#### to a file.  

    cat(sprintf("%.3f,%d,%d,%.1f\n", t, tn, nrow(sp), RSS$patches$area[attr(sp, "index")[1]]), file=MYFILE)
  }

This prints:

• the timestamp t
• the track number tn
• the number of rows in the sp matrix, which is the number of samples in the blip
• looks up the blip area in the vector RSS$patches$area.  This is a bit tricky: the expression

attr(sp, "index")[1] 

returns the patch number for the blip; that is, the index of its row in the RSS$patches data.frame for the current sweep.
This is used to look up the appropriate slot of RSS$patches$area, which is the vector of areas for each patch in the current sweep.
(Patches include both filtered and unfiltered blips; or more precisely, blips are patches which survived the current filtering.)

You can examine RSS$patches[] in the console when radR is paused on a blipmovie sweep to see what fields it contains.

Here is an example which instead plots the 2-D shape of each blip in each track:

  #### plot the 2-d shape in space of each blip in a track
  
  blipfun = function(sp, v, t, tn) {
    ## plot 2-d shape of blip in space
    try (
      plot(   gui.tx.matrix.to.spatial(sp)[,1:2],
              xlab="x",
              ylab="y",
              main=sprintf("At %s (track %d) blip has %d pixels\n", 
              strftime(as.POSIXlt(structure(t,class="POSIXct")), "%Y-%m-%d %H:%M:%OS3"),
              tn,
              nrow(sp)
           ))
      )
  }

Here's an explanation.

• the "try()" expression surrounding the main part of the body catches any errors that might occur.  Without this, the blip processing will stop the first time an error occurs
• the function gui.tx.matrix.to.spatial converts an nx2 matrix of sample/pulse coordinates to an n x 3 matrix of x,y,z coordinates; here, we use only the x and y columns (hence the [, 1:2] expression)
• nrow(sp) is the number of pixels in the blip
• the timestamp for the blip is numeric.  We convert it to class "POSIXct" by doing  structure(t,class="POSIXct"), then convert that
  to POSIXlt using as.POSIXlt, then finally, we format the date with strftime using codes that give year, month, day, hour, minute, seconds with 3 decimal digits

This plots a circle at the x, y coordinate of each pixel in a blip.

This example outputs a record for each 'pixel' (sample, really) in every track blip:

## for each pixel in each blip in a track, dump a record with these fields:

## ts      numeric timestamp
## trackNo track number from original track file
## blipID  arbitrary but unique ID for each different blip (unites pixels from same blip)
## range   range of this pixel (metres)
## azi     compass angle of this pixel (degrees clockwise from true North)
## phi     elevation angle of this pixel (degrees above horizon)
## int     intensity of this pixel (0..4095 for Seascan and USRP data; 0..255 for XIR3000 data)

blipID <<- 0

blipfun = function(sp, v, t, tn) {
  ## bump up blip counter; serves only to unite records from pixels in the same blip
  blipID <<- blipID + 1

  ## convert sample/pulse coordinates to spherical coordinates, given current radar scaninfo
  sph = rss.sp.to.sph(sp)
 
  cat(paste(sprintf("%.3f,%d,%d,%.1f,%.2f,%.1f,%.4f\n", t, tn, blipID, sph[,1], sph[,2], sph[,3], v), collapse=""), file=MYFILE)
}