$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ Migrating From Previous Versions of Khoros
E.1 How do I migrate my Khoros 1 software to Khoros 2?
The Migration toolbox provides a number of programs that will help you in migrating Khoros 1 code to Khoros 2. See the Migration Manual or the man pages for these routines to find out how to use them. The key programs included in the Migration toolbox are:
kfix Performs any possible updates of source code from Khoros 1 library calls to Khoros 2
kfixpane Updates a Khoros 1 *.pane file to a Khoros 2 *.pane file
uisupdate Updates a Khoros 1 *.form file to a Khoros 2 *.form file
mkproto Creates ANSI prototypes for functions
kdarwin An application that provides a front end to the programs listed above (and others), in order to help evolve a Khoros 1 program to a Khoros 2 software object.
The conversion programs available in the Migration toolbox aid in the migration of K1 software to K2 software, but they cannot do everything; depending on the contents of your Khoros 1 toolbox, you may have to do more work or less work to upgrade your software. The following summaries give an idea of the migration process for each category:
Cantata Workspaces
There is no automatic conversion. You must start Cantata with an empty workspace, and recreate the K1 workspace using the K2 operator equivalents. The reason for this is that all the operators in Cantata have been rewritten under 2; names and arguments have changed so much that a migration script would have been extremely difficult to write, and results would have been uncertain at best. Also, Cantata itself has change considerably since Khoros 1.
Xvroutines
There is a bit of automatic conversion provided. Use uisupdate to convert *.form and *.pane to a Khoros 2 version, use the K2 software development tools to generate the program framework, and finally, re-code the application by hand. Kfix can help to upgrade Khoros 1 function calls and bsd/non-POSIX.1 OS calls to their K2 equivalent. Since the interface to the GUI and visualization capabilities of Khoros 2 is so different from that of Khoros 1, a migration script would have been extremely difficult to write, and results would have been uncertain at best. Also, the functionality provided by GUI and Visualization is much more comprehensive than that which was provided in Khoros 1.
Vroutines
Most of the process can be done with automatic conversion. Use kdarwin to convert the K1 vroutine to a K2 kroutine. Kdarwin takes a Khoros 1 *.prog and *.pane, and creates a Khoros 2 kroutine software object. This type of conversion is the most standardized.
Libraries
The amount of automatic conversion possible varies with library contents. If the files are l*.c's from Khoros 1 vroutines, simply use kdarwin on each vroutine (see above). Otherwise, you must create an empty library software object with the Khoros 2 Craftsman, and upgrade the code by hand. Kfix can help to move old Khoros 1 function calls and bsd/non-POSIX.1 OS calls to their K2 equivalent.
E.2 How do I migrate my Khoros 2.0.x software to Khoros 2.1?
You will need to run kmigrate (found in the MIGRATION toolbox) for any toolboxes you have developed with an earlier version of Khoros 2 in order to update them to use the improved Imake rules and API changes. In most cases, this will fully update your toolboxes and objects to function in the new system.
Warning:
You should not run kmigrate on any of the toolboxes in
this release (i.e. those delivered with Khoros 2.1).
It is a good idea to make backup copies of your toolboxes before running kmigrate. If there is a problem with your toolbox that kmigrate can not recover from, you will need your backup copy so that you can fix the problem in the original toolbox and rerun kmigrate.
An example of how to run kmigrate is:
> kmigrate -tb <toolbox1>,<toolbox2>,<toolbox3>
Note: Spaces are not allowed between toolbox names.
Some caveats to migration:
E.3 VIFF Khoros format replaced by Khoros Data Format (KDF)
Background
The Khoros 2.1 release contains a new standard file format for the Khoros system, the Khoros Data Format (KDF). This format is intended to replace the Khoros 2.0.x Viff format.
The K2 Viff was a modified gdbm database capable of storing data segments as well as arbitrary attributes. The format was too complex to be documented, given the dependence on the gdbm database, meaning that reading the Viff files outside of the Khoros system was not possible. Furthermore, there was too much overhead associated with the gdbm code, making this format one of our least efficient ones.
The new KDF format was designed to solve all these problems. It is as structurally flexible as the old K2 Viff, and is well defined. Efficiency and simplicity were the driving factors in its design. It has proven to be much more maintainable, and much more efficient than the K2 Viff was.
The Khoros 1 Xvimage format is now again called Viff. The old Khoros 2 VIFF has been renamed to Gdbm and is still supported for input.
Note:
Please use the kformats program in DATAMANIP to begin
migrating your Gdbm data files to KDF.
With data services, we've tried to abstract away the notion of file formats. Instead, we encourage people to access data in terms of the segments and attributes of a data model. You access your data with data services library calls which operate on an abstract data object. The data object can then be stored in any supported format.
Basically, the K2 VIFF was a database capable of storing arbitrary data segments of any dimensionality, size, and index order as well as arbitrary attributes. There is a lot of overhead associated with the K2 Viff, and not much of the system can use its flexibility. The overhead and flexibility is tied to the use of using the dbm access routines in which to store and retrieve data.
We have designed a new format which is structurally flexible (like the K2 Viff), but is more structured in terms of its storage (like the K1 Xvimage). We wanted something that was a cross between xvimage and viff.
Some of the design objectives for this format :
Flexible
Since data services can store arbitrary attributes and segments which means fixed formats (like xvimage, rast, xwd, gif) will not work. We need to have a format that can completely represent all attributes and segments.
Efficient
The K2 Viff is not very efficient. The other formats are fairly efficient but not flexible. For data processing routines that take a long time this is not really important, but for light weight processes that are processing small data sets, the overhead associated with Viff becomes fairly substantial. This becomes particularly noticeable in Cantata.
Maintainable
The K2 Viff has had some problems with machine compatibility and with memory management. We wanted a simple format which is easy to maintain. The dbm code has exhibited some memory leaks which have not been trackable.
Streamable
We wanted a format that can be streamed. None of the existing formats can be streamed. The K2 Viff is flexible, but its dependence on dbm and its random access capability means that it can't be streamed.
Large Data Sets
Only xvimage and viff were written so that they could do partial reads and writes. So another criteria was that the format be able to support partial reads and writes.
Minimal Storage Overhead
When storing small data sets both xvimage and viff have a minimal header size, even if the data set is to store a single value data point. To store a single byte (value only) data set created using kgconst the sizes are:
kdf: 173 bytes
viff: 1025 bytes
gdbm: 4379 bytes
KDF Format Specification
The Big Picture
The format for a KDF file consists of a header, a series of attribute blocks, and a series of data blocks. It can be pictured roughly as follows:
KDF Header Attribute Block for Object Attribute Block for Segment 1 Attribute Block for Segment 2 ... Data Segment 1 Data Segment 2 ...
KDF Header
The KDF header consists of 7 bytes followed by three integers. Note that all information contained in a KDF file will be stored in a machine specific format. The seventh byte indicates the machine storage type for the particular file.
4 bytes 2 bytes 1 byte 1 integer 1 integer magic # version # machine type # of data # of attr. sets blocks
The magic number and version number: 01 03 19 94 00 02
The machine type follows the numbers defined in: $BOOTSTRAP/include/machine/kmachine.h
The number of data sets is currently hardwired to 1. This will be used in the future to store multiple data objects per file.
There is always at least one attribute block for storing
the object level attributes. Beyond that, there is one attribute
block for each segment in a data object. The attribute blocks
follow sequentially from the header.
Attribute Blocks
The format for the beginning of an attributes block is:
1 NULL-terminated string 1 integer segment name number of attributes for segment
The segment name will simply be a NULL ('\0') for the object level attribute block. The number of attributes describes the number of user-defined attributes for a segment and may be 0.
If the segment name is NULL, then none of the following information will be present. If the segment name is not NULL, then all the physical segment characteristics will be stored next. The segment characteristics are the dimension, datatype, size, and index order. They are stored as follows:
1 integer 1 NULL-terminated string dimension datatype
dimension integers for size size[0] size[1] ... size[dim-1]
dimension integers for order index order[0] index order[1] ... index order[dim-1]
1 integer 1 integer These should be hardwired to -1 fixed dimension dimension index for now. In the future, these fields will be used for storing multi-resolution data.
Datatypes are stored as a string representation of the datatype. The routine kdefine_to_datatype can be used to convert from the standard Khoros type identifiers to strings and the routine kdatatype_to_define can be used to convert the other direction. (Note: These routines may not be available in Khoros 2.0.2)
Valid datatypes are: bit, byte, unsigned byte, short, unsigned short, integer, unsigned integer, long, unsigned long, float, double, complex, and double complex.
Following this will be all other attributes for this segment. There will be exactly the number of attributes as was indicated at the top of the attribute block. Each attribute will follow the format described in the next section.
attribute 0 attribute 1 ...
Attributes
The format for attributes is:
1 string 1 integer 1 integer 1 string attribute name # of arguments argument size datatype attribute data end of attr. (eoa) tag
The attribute name is a string corresponding to the name of the attribute. The number of attribute arguments and the size of each arguments follow. The datatype of the attribute is stored next. Datatypes are stored in a string. The routine kdefine_to_datatype can be used to convert from the standard khoros type identifiers to strings and the routine kdatatype_to_define can be used to convert the other direction. (Note: These routines may not be available in Khoros 2.0.2)
The valid standard datatypes are: bit, byte, unsigned byte, short, unsigned short, integer, unsigned integer, long, unsigned long, float, double, complex, double complex, and string. Any arbitrarily-defined datatype is also allowed, although it is dependent on the datatype being defined in order to be read.
The attribute data follows, stored in a large contiguous block. The attribute data is followed by an end of attribute (eoa) tag.
The end of attribute tag is a string: "<>" . This tag is only really used when reading past attributes of an unknown data type (an undefined structure datatype for instance).
Data Segments
The data segments are stored following the attribute blocks in a series of blocks with all the data for that particular segment. There is one block per segment. The order of the data segment blocks must match the order of the corresponding attribute blocks.
data segment 1 data segment 2 ...
The amount of data present for each segment must be consistent with the physical segment characteristics specified in the attribute block that corresponds to this segment.
For instance, a segment with the characteristics of dimension = 2, size = [32, 24], and datatype = "byte," dataset would be expecting 32x24 bytes of data to be stored for its segment data block.
$
$BOOTSTRAP
Location of the bootstrap toolbox; not an environment
variable that you should set.
$DATAMANIP
Location of the datamanip toolbox; not an environment
variable that you should set.
$DATASERV
Location of the dataserv toolbox; not an environment
variable that you should set.
$DESIGN
Location of the design toolbox; not an environment
variable that you should set.
$ENVISION
Location of the envision toolbox; not an environment
variable that you should set.
$GEOMETRY
Location of the geometry toolbox; not an environment
variable that you should set.
$IMAGE
Location of the image toolbox; not an environment
variable that you should set.
$MIGRATION
Location of the migration toolbox; not an environment
variable that you should set.
$SAMPLEDATA
Location of the sampledata toolbox; not an
environment variable that you should set.
$SUPPORT
Location of the support toolbox; not an environment
variable that you should set.
A
alias
An alias created in the Khoros system is a way to define a
shorthand for input file locations. For example, a file called
~scientist/data/images/chest.mri.2103x3000.short could
be aliased to image:chest1 .
animate
Application for interactive animation of image sequences.
API
Application Programming Interface
AthenaIncludedir
The variable in the Site.arch that indicates where
the Athena widget set includes are located
AthenaLibdir
The variable in the Site.arch that indicates where
the Athena widget set library is located
B
Basic Services
Utilities located in the kutils library of the bootstrap toolbox,
including a large number of utility functions to handle memory
allocation, string manipulation, string parsing, and message reporting.
BinDir
The variable in the Site.arch that indicates the directory
for Khoros binaries.
BOOTSTRAP
Lowest-level toolbox, containing base Khoros utilities such
as installation script, Imakefile/Makefile generators, and foundation
services libraries
C
Cantata
Khoros visual language; an advanced data flow visual language
that supports coarse grain distributed processes and both stream
and block data. Its visual hierarchy, iteration, flow control,
and expression based parameters make it a powerful simulation
and prototyping system.
CLUI
see Command Line User Interface
Command Line User Interface
The command line user interface (CLUI) is supported by all
Khoros programs.
Conductor
Automatic code generation tool for generating code for interactive
X Window System based Khoros applications
Craftsman
Software object editor; allows you to modify, debug, compile,
and document software objects.
D
Data Services
Routines located in the kdataman, kappserv, and kdatafmt libraries
of the dataserv toolbox, comprising a powerful system for accessing
and manipulating data independent of format or type.
DATAMANIP
Toolbox containing data manipulation routines.
DATASERV
Toolbox containing data services
DESIGN
Toolbox containing software development system, visual language,
GUI & visualization services.
DoFullWarn
The variable in the Site.arch that indicates whether
all the compiler warnings should be displayed
when compiling
E
Editimage
Interactive image display, examination, and manipulation tool.
ENVISION
Toolbox containing data visualization applications.
Expression Services
A symbolic expression parser located in the kexpr library
of the bootstrap toolbox, which may be used to evaluate
mathematical equations and functions.
F
Foundation Services
A term used to encompass several distinct program services;
together, these services fulfill all the requirements of the Khoros
software infrastructure that do not deal with either data I/O
or data display. Foundation services includes Basic, Expression,
Math, Operating System, Software, and User Interface Services.
G
GEOMETRY
Toolbox containing 3D vector/polygon based visualization routines.
ghostwriter
Automatic code generation tool for generating code and documentation
for all Khoros programs.
Graphical User Interface
The graphical user interface (GUI) utilizes the X Toolkit
and one of the supported widget sets. Every program in Khoros
has a graphical user interface; xvroutines have a graphical
user interface of their own, which is displayed when they are
executed from the command line; kroutines have a graphical
user interface which is only displayed when they are accessed
via Cantata, or when their graphical user interface is
deliberately displayed with preview or Guise.
GUI
See Graphical User Interface
GUI & Visualization
Routines provided by all the libraries in the design
toolbox, taken together. They include functions to create and
maintain a GUI, plus a large variety of visual objects which can
be used to display images, graphics, plots, etc.
Guise
Direct manipulation GUI design tool.
H
HasAthena
The variable in the Site.arch that indicates whether
the system has the Athena widget set.
HasCenterLineC
The variable in the Site.arch that indicates whether
to use clcc to compile the code.
HasCodeCenter
The variable in the Site.arch that indicates whether
the system has the program codecenter.
HasCplusplus
The variable in the Site.arch that indicates whether
the system has a C++ compiler.
HasFortran
The variable in the Site.arch that indicates whether
the system has a FORTRAN compiler.
HasGcc
The variable in the Site.arch that indicates whether
to use gcc to compile the code
HasGcc2
The variable in the Site.arch that indicates whether
to use gcc 2.x or higher to compile the code.
HasLargeTmp
The variable in the Site.arch that indicates whether
the system has a large TMPDIR or /tmp.
HasMotif
The variable in the Site.arch that indicates whether
the system has the Motif widget set.
HasOlit
The variable in the Site.arch that indicates whether
the system has the Olit widget set.
HasPureCov
The variable in the Site.arch that indicates whether
the system has the program purecov.
HasPurelink
The variable in the Site.arch that indicates whether
the system has the program purelink.
HasPurify
The variable in the Site.arch that indicates whether
the system has the program purify.
HasQuantify
The variable in the Site.arch that indicates whether
the system has the program quantify.
HasSBrowser
The variable in the Site.arch that indicates whether
the system has the SPARCworks
development software.
HasSentinel
The variable in the Site.arch that indicates whether
the system has the program sentinel.
HasSharedLibraries
The variable in the Site.arch that indicates whether
the system supports shared libraries.
HasTestCenter
The variable in the Site.arch that indicates whether
the system has the program testcenter.
I
IMAGE
Toolbox containing image processing routines.
K
karith1
Command to performs single-operand pointwise arithmetic operations.
karith2
Command to performs dual-operand pointwise arithmetic operations.
kbuild
Script to compile all libraries, programs, and documentation
from the Khoros source code in your installation directory
(This may take a long time.) Provided for use on non-supported
platforms.
kconfigure
Script to set up user environments after installation.
kfft
Command to performs Fast Fourier Transforms (both forward
and reverse).
kgenimake
Program to automatically generate an architecture-independent
Imakefile for a software object.
kgenmake
Program to automatically generate an architecture dependent
Makefile from an Imakefile; correct & current Makefiles are
required by make to compile a software object.
kinstall
Script to copy, uncompress, and unpack all distribution
files contained on the CD-ROM into your installation directory.
Khoros Consortium
A collaboration between universities, industry, government,
and Khoral Research, Inc. to advance scientific software.
.khoros_env
The .khoros_env file typically contains settings
for Khoros environment variables.
kman
Command to access to Khoros on-line help in the form of man
pages.
L
LibDir
The variable in the Site.arch that indicates the directory
for Khoros libraries
M
make
Program to compile a software object.
Math Services
Utilities located in the kmath library of the bootstrap
toolbox, including machine independent implementations of common
mathematical operations; offers a variety of useful extensions
to the standard mathematical functions; designed to be highly
portable and efficient.
MIGRATION
Toolbox containing utilities to migrate from Khoros 1.0.5
to Khoros 2.0, and from Khoros 2.0.x to Khoros 2.1.
MotifIncludedir
The variable in the Site.arch that indicates where
the Motif widget set includes are located
MotifLibdir
The variable in the Site.arch that indicates where
the Moth widget set library is located
O
OlitIncludeDir
The variable in the Site.arch that indicates where
the Olit widget set includes are located
OlitLibDir
The variable in the Site.arch that indicates where
the Olit widget set library is located
Operating System Services
Routines found in the kutils library of the bootstrap
toolbox, which isolate Khoros from the operating system, and extend
the capabilities of the operating system to support distributed
computing.
S
SAMPLEDATA
Toolbox containing sample data for use with the Khoros software.
SentinelLibdir
The variable in the Site.arch that indicates where
the system has the sentinel libraries
software services
Routines located in the kcms library of the bootstrap toolbox,
designed to give the toolbox programmer a coherent view of the
various components of a toolbox or software object.
spectrum
Application for interactive signal/image classification.
SUPPORT
Toolbox containing various & sundry software development
utilities
T
Toolbox
A collection of programs and/or libraries that are managed
as an entity.
U
UIS
User Interface Specification file which define a command line
(CLUI) and/or graphical (GUI) user interface for Khoros programs.
User Interface Services
Private routines located in the kforms, kcodegen, and kclui
libraries that support UIS interpretation, CLUI support, code
generation, documentation generation, and UIS management.
V
visual language
A naturally visible approach to problem solving in which a
visual program is described as a directed graph, where each node
represents an operator or function and each directed arc represents
a path over which data flows.
W
Widget
X Window System terminology for a Graphical User Interface
component, such as a push-button, scroll-bar, or dialog box.
WidgetSet
The variable in the Site.arch that indicates which
widget set to use by default to compile xvroutines
X
X11IncludeDir
The variable in the Site.arch that indicates where
the X11 includes are located
X11LibDir
The variable in the Site.arch that indicates where
the X11 libraries are located
xprism
Application for 2D and 3D plotting.