Structures ToolThe Structures tool enables analyzing and editing of byte arrays based on user-creatable structure definitions, which can be built from arrays, unions, primitive types and enum values.
It has an own settings dialog, which can be reached by using the button. There are various options that can be configured, like the style (decimal, hexadecimal or binary) in which the values are displayed. Moreover it is possible to choose which structure definitions get loaded and which structures are shown in the view.
Structures are defined in Okteta Structure Definition files (based
on XML, with the file extension .osd).
Additionally a .desktop file contains
metadata about that structure description file, such as author, homepage
and license.
Currently there is no built-in support for creating or editing structure definitions, therefore this must be done manually as described in the next sections.
The easiest way of installing new structure definitions is by using the built-in KNewStuff support in Okteta. To install an existing structure open the settings dialog of the Structures tool. There select the Structures Management tab and press the button. The dialog that shows up now allows you to install and uninstall structures.
The Structures tool looks for structure descriptions in the
subdirectory okteta/structures/
of the user's directory for program data (find that by executing
qtpaths ).
You may need to create this directory if there are no structure
definitions installed yet.--paths GenericDataLocation
Two files exist for every structure definition: One file for the actual definition and a
.desktop file for the metadata (author, version, etc.).
In that directory there is a subdirectory for each structure
definition, which contains both the .desktop
file and the .osd or
main.js file of that definition.
For example, with the program data directory
qtpaths --paths GenericDataLocationokteta/structures/ExampleStructure
which contains a file ExampleStructure.desktop and a
file ExampleStructure.osd.
After having installed a new structure definition you need to restart Okteta before you can use it. Once Okteta has started, open the settings dialog of the Structures tool. There select the Structures Management tab and make sure the relevant structure definition is checked. Then switch to the Structures tab and make sure the desired element is listed on the right-hand side.
For common structures you may not need to create a definition yourself, but instead can reuse an already existing definition from places like store.kde.org.
You also may want to share a definition yourself. To do so, create a
file archive (e.g. a zipped tar archive, .tar.gz)
containing just the subdirectory with the .desktop
file and the structure definition file. Looking at the example in the last section this
would be the directory ExampleStructure with all its contents.
Using this format for sharing the structure definitions allows installing them inside Okteta and
requires no manual installation.
Note
A more up to date, but not completed guide to writing structure definitions can be found on the KDE UserBase Wiki.
There are two different ways of creating structure definitions. The first is writing the definition in XML the other is using JavaScript. The JavaScript approach allows you to create more complex structures with features like e.g. validating the structure. Using XML gives you less features but if a static structure is all you need this may be the easiest approach. If you need a dynamic structure i.e. where array lengths depend on other values in the structure or the structure layout is different when some member value changes, then you will have to write the structure definition in JavaScript. There is one exception to that rule: if you have an array where the length is supposed to be exactly the same as another value in the structure, then you can use XML. But if it is something like length - 1 it has to be JavaScript.
Note
A more up to date, but not completed guide to writing structure definitions can be found on the KDE UserBase Wiki.
The .osd XML file has one root element: <data> with no
attributes. Inside this element there must be one of the following elements:
- <primitive>
To create a primitive data types like e.g. int and float. This element accepts no subelements and can have the following attributes:
- type
The type of this primitive type. It must be one of the following:
char for a 8 bit ASCII character
int8, int16, int32, int64 for a signed integer of that size
uint8, uint16, uint32, uint64 for an unsigned integer of that size
bool8, bool16, bool32, bool64 for an unsigned boolean (0 = false, any other value = true) of that size
float for a 32 bit IEEE754 floating point number
double for a 64 bit IEEE754 floating point number
- <bitfield>
To create a bitfield. This element accepts no subelements and can have the following attributes:
- width
The number of bits used by this bitfield. Must be between 1 and 64.
- type
The type of this bitfield. It must be one of the following:
unsigned for a bitfield where the value will be interpreted as an unsigned value (value range from 0 to 2width - 1)
signed for a bitfield where the value will be interpreted as a signed value (value range from -2width - 1 to 2width - 1 - 1)
bool for a bitfield where the value will be interpreted as a boolean value
Note
Always remember to add padding after a <bitfield>, since otherwise the next element (except for strings and arrays, since they add padding automatically) will start in the middle of a byte. Obviously padding is not necessary if you want this behavior.
- <enum>
To create a primitive type, but where the values are displayed as members of an enumeration if possible. This element accepts no subelements (however you will need an <enumDef> tag in the file to reference it). It has the following attributes:
- enum
The underlying enum for this value. Must match the name attribute of one of the <enumDef> tags in this file.
- type
The type of this enum. See type attribute of <primitive>. Only difference is that Double and Float make no sense.
- <flags>
This is the same as <enum> with the only difference being that values are represented as a bitwise-or of all the values of the enumeration.
- <struct>
To create a structure. All other elements (including <struct>) can be a child of this and will then be part of the resulting structure
- <union>
To create a union. Basically the same as <struct> except for the fact that all child elements will start from the same offset. Useful for interpreting the same sequence of bytes in various ways.
- <array>
To create an array. This element accepts exactly one child (the underlying type of the array), which can be any of the elements, even <array> itself. It also has the following attributes:
- length
The number of elements in this array as a decimal number. Alternatively it can also be a string which matches the name attribute of a previously defined <primitive>, <enum> or <flags> element. Then the length will be the value of that element. Currently it is limited to 10000, because larger arrays would use too much memory and slow down the tool too much.
- <string>
To create a string in various encodings. By default you get a NULL- terminated C-style string. However different types of string can be created with the following attributes:
- terminatedBy
This attribute determines what unicode codepoint the string is terminated by. It must be a hexadecimal number (optionally with leading 0x). When encoding is ASCII, only values up to 0x7f are meaningful. If neither this nor maxCharCount nor maxByteCount are set, this is assumed to be set to 0 (C-style string)
- maxCharCount
The maximum number of chars this string can have. If terminatedBy is set too then whatever is reached first terminates the string. This is mutually exclusive with maxByteCount
- maxByteCount
The maximum number of bytes this string can be long. If terminatedBy is set too then whatever is reached first terminates the string. This is mutually exclusive with maxCharCount. With encodings like ASCII this is the same as maxCharCount.
- type
The encoding of this string. Can be one of the following:
ASCII
LATIN-1
UTF-8
UTF-16-LE or UTF-16-BE. If neither -LE or -BE suffix is given, little endian is assumed.
UTF-32-LE or UTF-32-BE. If neither -LE or -BE suffix is given, little endian is assumed.
Every element also accepts an attribute name which is then visible in the structures view.
Note
A more up to date, but not completed guide to writing structure definitions can be found on the KDE UserBase Wiki.
Our metadata file looks like this:
[Desktop Entry]
Icon=arrow-up❶
Type=Service
ServiceTypes=KPluginInfo
Name=Simple test structure
Comment=A very simple test structure containing only two items
X-KDE-PluginInfo-Author=Alex Richardson
X-KDE-PluginInfo-Email=foo.bar@email.org
X-KDE-PluginInfo-Name=simplestruct
X-KDE-PluginInfo-Version=1.0
X-KDE-PluginInfo-Website=https://www.plugin.org/
X-KDE-PluginInfo-Category=structure
X-KDE-PluginInfo-License=LGPL
X-KDE-PluginInfo-EnabledByDefault=false
The icon displayed in Okteta for this structure can be anything found by executing
|
These fields should all be pretty much self-explanatory, except for
X-KDE-PluginInfo-Name. The value of this field must match the name of the directory containing the file as well as the name of the .desktop file. When creating XML structure definitions the name of the .osd file must also match the name.
In this example we would have a directory named simplestruct
containing the file simplestruct.desktop. When defining structures in XML
the directory would also contain a file named simplestruct.osd. Using JavaScript we would have a file named main.js instead.
To start we create a definition for a very simple test structure containing only integral data types (one char, one 32-bit signed integer, and a bitfield). This would be expressed in C/C++ as:
struct simple {
char aChar;
int anInt;
bool bitFlag :1;
unsigned padding :7;
};
The first step is writing the .osd file according to the file format defined
in the previous section. We will call simplestruct.osd:
<?xml version="1.0" encoding="UTF-8"?>
<data>
<struct name="simple">
<primitive name="aChar" type="Char"/>
<primitive name="anInt" type="Int32"/>
<bitfield name="bitFlag" type="bool" width="1"/>
<bitfield name="padding" type="unsigned" width="7"/>
</struct>
</data>
which is fairly similar to the C/C++ definition.
Now create a directory simplestruct
inside the structure installation directory (see manually installing structure definitions)
and copy the two files to this directory. Now you can restart Okteta and use the new structure.
To implement the structure above in JavaScript, create a file named main.js instead of simplestruct.osd
and change X-KDE-PluginInfo-Category=structure to X-KDE-PluginInfo-Category=structure/js.
The contents of that file should be:
function init() {
var structure = struct({
aChar : char(),
anInt : int32(),
bitFlag : bitfield("bool", 1),
padding : bitfield("unsigned", 7),
})
return structure;
}
The structure displayed by Okteta is always the return value of the init function.
The following functions can be called to create a primitive type:
char()
int8(), int16(), int32() or int64()
uint8(), uint16(), uint32() or uint64()
bool8(), bool16(), bool32() or bool64()
float()
double()
The bitfield function takes two parameters, the first being a string consisting of
bool, signed or unsigned. The second parameter is an integer which sets the width in bits.
Next we create a definition of a more complex structure which we will
call "complex" and save in a file named complex.osd.
This structure will contain two arrays (one with fixed length and one
where the length is determined at runtime) as well as a nested structure
and a union.
<?xml version="1.0" encoding="UTF-8"?>
<data>
<struct name="complex">
<primitive name="size" type="UInt8" />
<union name="aUnion">
<array name="fourBytes" length="4">
<primitive type="Int8" />
</array>
</union>
<struct name="nested">
<array name="string" length="size"> <!-- references the field size above -->
<primitive type="Char" />
</array>
</struct>
</struct>
</data>
This would correspond to the following in pseudo-C/C++
struct complex {
uint8_t size;
union aUnion {
int8_t fourBytes[4];
};
struct nested {
char string[size] //not valid C++, references value of the uint8 size
};
};
Note
You can obviously only have dynamic length arrays reference fields before the array.
Next we create the complex.desktop file just as
in the example before (make sure you set X-KDE-PluginInfo-Name
correctly) and also do the same to install both files.
A few example structure definitions can be found in the Git repository.
This includes for example the file header for PNG files and the ELF file header.
An XML schema describing the structure of the .osd file can be found here.
If more information is needed feel free to contact me at (arichardson.kde gmail.com)