Stripping Text From Quasar DAT Files

Had some success today with stripping out floating point values from the DAT text files that BCO provided. The values are extracted through two functions. The first sets a patterns to find a specific value in the DAT file such as “10.00” and then does a second search to get another value in a offset column of text in a row marked by the value “10.00”. When the value is identified it is passed into an array. The second function then does the straightforward work of clearing the value of white space. The next stage is now to normalise this value¬† so it can be used as an animation key as described in, “Prototype Animation Anticipating Data from BCO” posted on 16th Feb last.

One unforeseeable issue that came up was related to the quantity of data. Pulling in text information dynamically to AS3 is typically unproblematic but in this case the smallest text file in the batch is42mb; other files are in excess of 213mb. An intermediate step may be required to copy out specific values from these raw files and generate new text files that are then passed to AS3.

A second issue is that when a selection of values is taken from the text file, that is one that represents the brightness value for a certain radius of pixels in the image from the telescope , there are in the smallest text file 21,511 values that represent changes in light readings from March to May 2008. I may therefore have to consider how to manage these values. Based on a very loose estimate of the data I have access to at present this would provide me with approx. half of one million keyframes generated from studies of quasars recorded over a period of one calendar year.

Before I go there I want to now invest some time with what I will do visually with the data. I intend soon return to looking at processes that will allow me to draw in real time with the information and extend earlier investigations as demonstrated in ‘Family Portrait’ (see post entitled “Real time Portraits” from 7th Feb last)