ISIS3 I/O

From DavinciWiki
Jump to: navigation, search

Davinci, with an I/O module and an appropriately installed and configured ISIS3 installation, has a the capability to read and write ISIS3 cubes. Headers, history, and some table data support are included.

Contents

Prerequisites

The ISIS3 module is currently in beta and not included in the build path of davinci, however it is in source control in the iomod_isis3 directory. This module has only been tested on 64-bit Linux systems (specifically CentOS 5.3 and CentOS 6.0) using ISIS 3.3 and 3.4. The 3.3 and 3.4 ISIS versions need different code. the build.sh will build the module for ISIS 3.3 and the build34.sh will build for ISIS 3.4. The resulting modules are called 'isis3.dvio' and 'isis34.dvio', respectively.

ISIS3 and QT4 (including the development headers for QT4) must be installed on your system and to compile the module, you must know where it is installed. Edit the appropriate build file in the distribution and change the values in ISIS3DIR and QTDIR to point to the proper locations. If you wish to build for 32 bit Linux systems, change the ARCH value to 32 as well. Other OSes are currently not supported, but will be in the future when the code to build it is distributed in core davinci. If you are using another operating system and/or compiler and you figure out how to compile it for it, please let us know! It will help get the build to be more portable.

Once you have edited this file to reflect the appropriate changes, Run it:

 ./build.sh

or

 ./build34.sh


If all is right, you should have a file in the directory called "isis3.dvio"

This file should be copied to where your davinci modules are kept. Alternatively, you can set the environment variable DV_MOD_PATH to the location of this file when you run davinci:

 DV_MOD_PATH=directory davinci

You must have your environment set up correctly to use ISIS3. Within the Mars Space Flight Facility, simply running

 $ isis3setup

will do this. Elsewhere, please consult the ISIS 3 documentation for how to do this for your environment.

When you start up davinci, you must load the IO module before it will work:

 dv> insmod("isis3")
 Initializing davinci/ISIS3 IO module version 0.9.0 (ISIS 3.3.x)
 dv>

If you don't have your ISIS3 environment set up properly, loading the module will give you a warning:

 dv> insmod("isis3")
 Initializing davinci/ISIS3 IO module version 0.9.0 (ISIS 3.3.x)
 Warning: ISIS Environment not setup correctly to use ISIS3 API.
 Please set your ISISROOT environment variable to the root of an ISIS3 install,
 or undesired operation may result.
 dv>

Simple Usage: Reading a cube

When the module successfully loads, the davinci built-in function load() now knows how to read ISIS3 files without you having to take any extra action:

 dv> a = load("/u/randy/isis3/I35449018RDR.cub")
 Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
 1024x5308x10 array of float, bsq format [217,415,680 bytes]
 dv>

Accessing Headers: load_pds

The module also supports loading the full ISIS3 structure via the load_pds function:

To read an ISIS3 cube with all headers and data, you can use the load_pds() function:


 dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub")
 Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
 struct, 3 elements
   cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
   IsisCube: struct, 9 elements
       Core: struct, 7 elements...
       Instrument: struct, 14 elements...
       Archive: struct, 10 elements...
       BandBin: struct, 5 elements...
       Kernels: struct, 14 elements...
       Mapping: struct, 17 elements...
       AlphaCube: struct, 9 elements...
       History: struct, 3 elements...
       isis_struct_type: "object"
   NaifKeywords: struct, 8 elements
       BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
       BODY_FRAME_CODE: 10014
       CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
       INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
       isis_struct_type: "object"

Note that this form does NOT include the table data. To get the table data, you must add the suffix_data=1 option to your load_pds() call:

 dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub", suffix_data=1)
 Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
 struct, 4 elements
   cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
   IsisCube: struct, 9 elements
       Core: struct, 7 elements...
       Instrument: struct, 14 elements...
       Archive: struct, 10 elements...
       BandBin: struct, 5 elements...
       Kernels: struct, 14 elements...
       Mapping: struct, 17 elements...
       AlphaCube: struct, 9 elements...
       History: struct, 3 elements...
       isis_struct_type: "object"
   TableList: struct, 4 elements
       InstrumentPointing: struct, 11 elements...
       InstrumentPosition: struct, 10 elements...
       BodyRotation: struct, 11 elements...
       SunPosition: struct, 10 elements...
   NaifKeywords: struct, 8 elements
       BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
       BODY_FRAME_CODE: 10014
       CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
       INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
       isis_struct_type: "object"

This form now returns a structure that includes a "TableList" member, in which all the relevant table data is stored.

Writing an ISIS3 cube

To write an ISIS3 cube, you can save an array or a structure. Saving an array will create a very minimal but correct ISIS3 cube. Saving a structure requires at a minimum for the structure to have an array member called "cube."

Structures that you want to be treated as something other than an ISIS3 object should have a member in them called "isis_struct_type" that contain the values "group", "history", or "object" (which is the default). Keep in mind that groups may only contain simple values and cannot contain other structures. For more information on these objects, consult the ISIS3 PVL documentation. Normally, though, this shouldn't concern you as you have most likely started from an ISIS3 object and virtually all of the data that concerns you is already pre-populated from the load_pds call.

Example:


 dv> write(a, "output.cub", type=isis3, force=1)
 Using ISIS3 API to write file 'output.cub'.
 Closing file...              
 dv> 

The write will look for the TableList member automatically, so if on the off-chance you want to write the cube back out WITHOUT the tables, you can just use the remove_struct() function:

 dv> remove_struct(a, "TableList")
 struct, 4 elements
   InstrumentPointing: struct, 11 elements
       J2000Q0: struct, 4 elements...
       J2000Q1: struct, 4 elements...
       J2000Q2: struct, 4 elements...
       J2000Q3: struct, 4 elements...
       AV1: struct, 4 elements...
       AV2: struct, 4 elements...
       AV3: struct, 4 elements...
       ET: struct, 4 elements...
       data: 8x91x1 array of double, bsq format [5,824 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   InstrumentPosition: struct, 10 elements
       J2000X: struct, 4 elements...
       J2000Y: struct, 4 elements...
       J2000Z: struct, 4 elements...
       J2000XV: struct, 4 elements...
       J2000YV: struct, 4 elements...
       J2000ZV: struct, 4 elements...
       ET: struct, 4 elements...
       data: 7x9x1 array of double, bsq format [504 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   BodyRotation: struct, 11 elements
       J2000Q0: struct, 4 elements...
       J2000Q1: struct, 4 elements...
       J2000Q2: struct, 4 elements...
       J2000Q3: struct, 4 elements...
       AV1: struct, 4 elements...
       AV2: struct, 4 elements...
       AV3: struct, 4 elements...
       ET: struct, 4 elements...
       data: 8x2x1 array of double, bsq format [128 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   SunPosition: struct, 10 elements
       J2000X: struct, 4 elements...
       J2000Y: struct, 4 elements...
       J2000Z: struct, 4 elements...
       J2000XV: struct, 4 elements...
       J2000YV: struct, 4 elements...
       J2000ZV: struct, 4 elements...
       ET: struct, 4 elements...
       data: 7x2x1 array of double, bsq format [112 bytes]
       cube_association: "None"
       isis_struct_type: "table"

dv>


 dv> write(a, "new.cub", type=isis3, force=1)
 Using ISIS3 API to write file 'new.cub'.
 Closing file...              
 dv>

remove_struct() returns the part of the structure that was deleted, so you can store that in a variable if you want to reattach the table information again after your write:


 dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub", suffix_data=1)
 Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
 struct, 4 elements
   cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
   IsisCube: struct, 9 elements
       Core: struct, 7 elements...
       Instrument: struct, 14 elements...
       Archive: struct, 10 elements...
       BandBin: struct, 5 elements...
       Kernels: struct, 14 elements...
       Mapping: struct, 17 elements...
       AlphaCube: struct, 9 elements...
       History: struct, 3 elements...
       isis_struct_type: "object"
   TableList: struct, 4 elements
       InstrumentPointing: struct, 11 elements...
       InstrumentPosition: struct, 10 elements...
       BodyRotation: struct, 11 elements...
       SunPosition: struct, 10 elements...
   NaifKeywords: struct, 8 elements
       BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
       BODY_FRAME_CODE: 10014
       CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
       INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
       isis_struct_type: "object"
 dv> temp = remove_struct(a, "TableList")
 struct, 4 elements
   InstrumentPointing: struct, 11 elements
       J2000Q0: struct, 4 elements...
       J2000Q1: struct, 4 elements...
       J2000Q2: struct, 4 elements...
       J2000Q3: struct, 4 elements...
       AV1: struct, 4 elements...
       AV2: struct, 4 elements...
       AV3: struct, 4 elements...
       ET: struct, 4 elements...
       data: 8x91x1 array of double, bsq format [5,824 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   InstrumentPosition: struct, 10 elements
       J2000X: struct, 4 elements...
       J2000Y: struct, 4 elements...
       J2000Z: struct, 4 elements...
       J2000XV: struct, 4 elements...
       J2000YV: struct, 4 elements...
       J2000ZV: struct, 4 elements...
       ET: struct, 4 elements...
       data: 7x9x1 array of double, bsq format [504 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   BodyRotation: struct, 11 elements
       J2000Q0: struct, 4 elements...
       J2000Q1: struct, 4 elements...
       J2000Q2: struct, 4 elements...
       J2000Q3: struct, 4 elements...
       AV1: struct, 4 elements...
       AV2: struct, 4 elements...
       AV3: struct, 4 elements...
       ET: struct, 4 elements...
       data: 8x2x1 array of double, bsq format [128 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   SunPosition: struct, 10 elements
       J2000X: struct, 4 elements...
       J2000Y: struct, 4 elements...
       J2000Z: struct, 4 elements...
       J2000XV: struct, 4 elements...
       J2000YV: struct, 4 elements...
       J2000ZV: struct, 4 elements...
       ET: struct, 4 elements...
       data: 7x2x1 array of double, bsq format [112 bytes]
       cube_association: "None"
       isis_struct_type: "table"
 dv> write(a, "output.cub", type=isis3, force=1)
 Using ISIS3 API to write file 'output.cub'.
 Closing file...              
 dv> a.TableList = temp
 struct, 4 elements
   InstrumentPointing: struct, 11 elements
       J2000Q0: struct, 4 elements...
       J2000Q1: struct, 4 elements...
       J2000Q2: struct, 4 elements...
       J2000Q3: struct, 4 elements...
       AV1: struct, 4 elements...
       AV2: struct, 4 elements...
       AV3: struct, 4 elements...
       ET: struct, 4 elements...
       data: 8x91x1 array of double, bsq format [5,824 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   InstrumentPosition: struct, 10 elements
       J2000X: struct, 4 elements...
       J2000Y: struct, 4 elements...
       J2000Z: struct, 4 elements...
       J2000XV: struct, 4 elements...
       J2000YV: struct, 4 elements...
       J2000ZV: struct, 4 elements...
       ET: struct, 4 elements...
       data: 7x9x1 array of double, bsq format [504 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   BodyRotation: struct, 11 elements
       J2000Q0: struct, 4 elements...
       J2000Q1: struct, 4 elements...
       J2000Q2: struct, 4 elements...
       J2000Q3: struct, 4 elements...
       AV1: struct, 4 elements...
       AV2: struct, 4 elements...
       AV3: struct, 4 elements...
       ET: struct, 4 elements...
       data: 8x2x1 array of double, bsq format [128 bytes]
       cube_association: "None"
       isis_struct_type: "table"
   SunPosition: struct, 10 elements
       J2000X: struct, 4 elements...
       J2000Y: struct, 4 elements...
       J2000Z: struct, 4 elements...
       J2000XV: struct, 4 elements...
       J2000YV: struct, 4 elements...
       J2000ZV: struct, 4 elements...
       ET: struct, 4 elements...
       data: 7x2x1 array of double, bsq format [112 bytes]
       cube_association: "None"
       isis_struct_type: "table"
 dv>

History

The IO module will automatically add a history object to the ISIS3 cube, called "davisis3". This will allow us to more readily identify when an ISIS3 cube has been pushed through davinci. This does not preclude you from adding your own history object to something you want to save. For example:

 dv> a = load_pds("/u/randy/isis3/I35449018RDR.cub", suffix_data=1)
 Using ISIS3 API to read file '/u/randy/isis3/I35449018RDR.cub'.
 struct, 4 elements
   cube: 1024x5308x10 array of float, bsq format [217,415,680 bytes]
   IsisCube: struct, 9 elements
       Core: struct, 7 elements...
       Instrument: struct, 14 elements...
       Archive: struct, 10 elements...
       BandBin: struct, 5 elements...
       Kernels: struct, 14 elements...
       Mapping: struct, 17 elements...
       AlphaCube: struct, 9 elements...
       History: struct, 3 elements...
       isis_struct_type: "object"
   TableList: struct, 4 elements
       InstrumentPointing: struct, 11 elements...
       InstrumentPosition: struct, 10 elements...
       BodyRotation: struct, 11 elements...
       SunPosition: struct, 10 elements...
   NaifKeywords: struct, 8 elements
       BODY499_RADII: 3x1x1 array of double, bsq format [24 bytes]
       BODY_FRAME_CODE: 10014
       CLOCK_ET_-53_944976590.153_COMPUTED: "9e44da08ebb3b241"
       INS-53031_TRANSX: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_TRANSY: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSS: 3x1x1 array of double, bsq format [24 bytes]
       INS-53031_ITRANSL: 3x1x1 array of double, bsq format [24 bytes]
       isis_struct_type: "object"
 
 dv> z = struct()
 struct, 0 elements
 dv> z.CodeVersion = "0.1"
 "0.1"
 dv> z.Description = "My custom davinci process on ISIS3 cubes."
 "My custom davinci process on ISIS3 cubes."
 dv> a.IsisCube.History.myprocess = z
 struct, 2 elements
   CodeVersion: "0.1"
   Description: "My custom davinci process on ISIS3 cubes."
 dv> write(a, "output.cub", type=isis3, force=1)
 Using ISIS3 API to write file 'output.cub'.
 Closing file...              
 dv> 

Then, upon inspecting the ISIS3 history object:

 $ cathist from=output.cub 
 Object = spiceinit
   IsisVersion       = "3.3.0.3646 | 2011-09-19"
   ProgramVersion    = 2011-10-05
   ProgramPath       = /mars/common/isis3/3.3beta_64/isis/bin
   ExecutionDateTime = 2011-11-01T17:16:35
   HostName          = Unknown
   UserName          = randy
   Description       = "Determine SPICE kernels for a camera cube"
   
   Group = UserParameters
     FROM         = ./I35449018RDR.raw.cub
     WEB          = false
     ATTACH       = TRUE
     CKSMITHED    = FALSE
     CKRECON      = TRUE
     CKPREDICTED  = FALSE
     CKNADIR      = FALSE
     SPKSMITHED   = FALSE  
     SPKRECON     = TRUE
     SPKPREDICTED = FALSE
     SHAPE        = SYSTEM
     STARTPAD     = 0.0
     ENDPAD       = 0.0
     URL          = http://services.isis.astrogeology.usgs.gov/cgi-bin/spicein-
                    it.cgi
     PORT         = 80
   End_Group
 End_Object
   
 Object = cam2map
   IsisVersion       = "3.3.0.3646 | 2011-09-19"
   ProgramVersion    = 2011-02-10
   ProgramPath       = /mars/common/isis3/3.3beta_64/isis/bin
   ExecutionDateTime = 2011-11-01T17:16:38
   HostName          = Unknown
   UserName          = randy
   Description       = "Convert camera image to a map projection"
  
   Group = UserParameters
     FROM         = ./I35449018RDR.raw.cub
     MAP          = $base/templates/maps/sinusoidal.map
     TO           = ./I35449018RDR.cub
     MATCHMAP     = false
     PIXRES       = CAMERA
     DEFAULTRANGE = MINIMIZE
     LONSEAM      = AUTO
     INTERP       = CUBICCONVOLUTION
   End_Group
 End_Object
 
 Object = myprocess
   CodeVersion = 0.1
   Description = "My custom davinci process on ISIS3 cubes."
 End_Object
 
 Object = davisis3
   DavinciVersion            = "daVinci Version #2.09"
   DavinciIsis3ModuleVersion = "0.9.0 (ISIS 3.3.x)"
   ExecutionDateTime         = 2012-08-29T00:08:01
   HostName                  = akagi
   UserName                  = randy
   Description               = "ISIS3 cube written from davinci."
 End_Object
 End
 $

Limitations

The table support is limited to simple homogeneous tables of the same data type. This is adequate for the NAIF/SPICE tables that are attached to spiceinit and should work for many other cases, such as "backplanes" of metadata, like latitude, longitude, solar azimuth, et cetera. ISIS3 vectors and mixed data type records are not yet supported until I can figure out a clean way to represent those within davinci data structures. This limitation allows us to use davinci arrays to store the table data, which is very efficient and meets our current use cases. This module will once again be revisited if and when use cases require more complicated table support.

Personal tools