_____
---' __\_______
______)
__) Release notes for poke 3.0
__)
---._______)
I am happy to announce a new major release of GNU poke, version 3.0.
This release is the result of a year of development. A lot of things
have changed and improved with respect to the 2.x series; we have
fixed many bugs and added quite a lot of new exciting and useful
features. See below for a description of many of them.
From now on, we intend to do not one but two major releases of poke
every year. What is moving us to change this is the realization that
users have to wait for too long to enjoy new features, which are
continuously being added in a project this young and active.
The tarball poke-3.0.tar.gz is now available at
https://ftp.gnu.org/gnu/poke/poke-3.0.tar.gz.
GNU poke (http://www.jemarch.net/poke) is an interactive, extensible
editor for binary data. Not limited to editing basic entities such
as bits and bytes, it provides a full-fledged procedural,
interactive programming language designed to describe data
structures and to operate on them.
Thanks to the people who contributed with code and/or documentation to
this release. In certain but no significant order they are:
Mohammad-Reza Nabipoor
Arsen Arsenović
Luca Saiu
Bruno Haible
apache2
Indu Bhagat
Agathe Porte
Alfred M. Szmidt
Daiki Ueno
Darshit Shah
Jan Seeger
Sergio Durigan Junior
... and yours truly
As always, thank you all!
But wait, this time we also have special thanks:
To Bruno Haible for his invaluable advise and his help in
throughfully testing this new release in many different platforms
and configurations.
To the Sourceware overseers, Mark Wielaard, Arsen Arsenović, and Sam
James for their help in setting up the buildbots we are using for CI
at sourceware.
What is new in this release:
User interface updates
======================
- A screen pager has been added to the poke application. If enabled
with the `.set pager yes' option, output will be paged one screenful
at a time.
- A tracer has been added to libpoke and the poke application. If
enabled with the `.set tracer yes' option, subsequent loaded Poke
types will be instrumentalized so calls to user-defined handlers are
executed when certain events happen:
+ Every time a field gets mapped.
+ Every time a struct/union gets mapped.
+ Every time a field gets constructed.
+ Every time a struct/union gets constructed.
+ Every time an optional field is omitted when mapping or
constructing.
This is very useful when debugging and writing pickles.
- A new command sdiff (for "structured diff") has been added to the
poke application, that provides a way to generate patchable diffs of
mapped structured Poke values. This command is an interface to the
structured diffs provided by the new diff.pk pickle.
- When no name is passed to the .mem command, an unique name for the
memory IOS with the form *N* will be used automatically, where N is
a positive integer.
- auto-completion of 'attributes is now available in the poke
application.
- Constraint errors now contain details on the location (which field)
where the constraint error happens, along with the particular
expression that failed.
- Inline assembler expressions and statements are now supported:
,----
| asm (TEMPLATE [: OUTPUTS [: INPUTS]])
| asm TYPE: (TEMPLATE [: INPUTS])
`----
where PVM assembly code can be executed as part of normal Poke
programs.
- Both `printf' and `format' now support printing values of type
`any'.
- Both `printf' and `format' now support printing integral values
interpreted as floating-point values encoded in IEEE 754. Format
tags %f, %g and %e are supported. This feature, along with the new
ieee754.pk pickle, eases dealing with floating-point data in binary
data.
- Pre-conditional optional fields are added to complement the
currently supported post-conditional optional fields. A
pre-conditional optional field like:
,----
| if (CONDITION)
| TYPE FNAME;
`----
makes FNAME optional based on the evaluation of CONDITION. But the
field itself is not mapped if the condition evaluates to false.
- A new option `.set autoremap no' can be used in order to tell poke
to not remap mapped values automatically. This greatly speeds up
things, but assumes that the contents of the IO space are not
updated out of the control of the user. See the manual for details.
- The :to argument to the `extract' command is now optional, and
defaults to the empty string.
- ${XDG_CONFIG_HOME:-$HOME/.config} is now preferred to
XDG_CONFIG_DIRS.
Poke Language updates
=====================
- Array and struct constructors are now primaries in the Poke syntax.
This means that it is no longer necessary to enclose them between
parenthesis in constructions like:
,----
| (Packet {}).field
`----
and this is now accepted:
,----
| Packet {}.field
`----
- Bit-concatenation is now supported in l-values. After executing
this code:
,----
| var a = 0 as int<4>;
| var b = 0 as uint<28>;
|
| a:::b = 0x12345678;
`----
the value of `a' is 0x1N and the value of `b' is (uint<28>)
0x2345678.
- Arrays can now be indented by size, by specifying an offset as an
index. This is particularly useful for accessing structures such as
string tables without having to explicitly iterate on the array's
elements.
- Union types can now be declared as "integral". The same features of
integral structs are now available for unions: integration,
deintegration, the ability of being used in contexts where an
integer is expected, etc.
- Support for "computed fields" has been added to struct and union
types. Computed fields are accessed just like regular fields, but
the semantics of referring to them and of assigning to them are
specified by the user by the way of defining getter and setter
methods.
- This version introduces three new Poke attributes that work on
values of type `any':
,----
| VAL'elem (N)
| evaluates to the Nth element in VAL, as a value of type `any'.
|
| VAL'eoffset (N)
| evaluates to the offset of the Nth element in VAL.
|
| VAL'esize (N)
| evaluates to the size of the Nth element in VAL.
|
| VAL'ename (N)
| attribute evaluates to the name of the Nth element in VAL.
`----
- Two new operators have been introduced to facilitate operating Poke
array as stacks in an efficient way: apush and apop. Since these
operators change the size of the involved arrays, they are only
allowed in unbounded arrays.
- Poke programs can now hook in the IO subsystem by installing
functions that will be invoked when certain operations on IO spaces
are being performed:
,----
| ios_open_hook
| Functions in this hook are invoked once a new IO space has been
| opened.
|
| ios_set_hook
| Functions in this hook are invoked once the current IO space
| changes.
|
| ios_close_pre_hook
| ios_close_hook
| Functions in these hooks are invoked before and after an IO space is
| closed, respectively.
`----
- The 'length attribute is now valid in values of type `any'.
- Poke declarations can now be annotated as `immutable'. It is not
allowed to re-define immutable definitions.
- A new compiler built-in `iolist' has been introduced, that returns
an array with the IO space identifiers of currently open IOS.
- We have changed the logic of the EXCOND operator ?!. It now
evaluates to 1 (true) if the execution of the first operand raises
the specified exception, and to 0 (false) otherwise. We profusedly
apologize for the backwards incompatibility, but this is way better
than the previous (reversed) logic.
- The containing struct or union value can now be refered as SELF in
the body of methods. SELF is of type `any'.
- Integer literal suffixes (B, H, U, etc) are case-insensitive. But
until now little-case `b' wasn't being recognized as such. Now `1B'
is the same than `1b'.
- Casting to union types now raise a compile-time error.
- If no explicit message is specified in calls to `assert', a default
one showing the source code of the failing condition is constructed
and used instead.
- An operator `remap' has been used in order to force a re-map of some
mapped Poke value.
- Signed integral types of one bit are not allowed. How could they
be, in two's complement?
- The built-in function get_time has been renamed to gettime, to
follow the usual naming of the corresponding standard C function.
Standard Poke Library updates
=============================
- New standard functions:
,----
| eoffset (V, N)
| Given a value of type `any' and a name, returns the offset of
| the element having that name.
|
| openset (HANDLER, [FLAGS])
| Open an IO space and make it the current IO space.
|
| with_temp_ios ([HANDLER], [FLAGS], [DO], [ENDIAN])
| Execute some code with a temporary IO space.
|
| with_cur_ios (IOS, [DO], [ENDIAN])
| Execute some code on some given IO space.
`----
libpoke updates
===============
- New API function pk_struct_ref_set_field_value.
- New API function pk_type_name.
Pickles updates
===============
- New pickles provided in the poke distribution:
,----
| diff.pk
| Useful binary diffing utilities. In particular, it implements
| the "structured diff" format as described in
| https://binary-tools.net/bindiff.pdf.
|
| io.pk
| Facilities to dump data to the terminal.
|
| pk-table.pk
| Convenient facilities to Poke programs to print tabulated data.
|
| openpgp.pk
| Pickle to poke at OpenPGP RFC 4880 data.
|
| sframe.pk
| sframe-dump.pk
| Pickles for the SFrame unwinding format, and related dump
| utilities.
|
| search.pk
| Utility for searching data in IO spaces that conform to some
| given Poke type.
|
| riscv.pk
| Pickle to poke at instructions encoded in the RISC-V instruction
| set (RV32I). It also provides methods to generate assembly
| language.
|
| coff.pk
| coff-aarch64.pk
| coff-i386.pk
| COFF object files.
|
| pe.pk
| pe-amd64.pk
| pe-arm.pk
| pe-arm64.pk
| pe-debug.pk
| pe-i386.pk
| pe-ia64.pk
| pe-m32r.pk
| pe-mips.pk
| pe-ppc.pk
| pe-riscv.pk
| pe-sh3.pk
| PE/COFF object files.
|
| pcap.pk
| Capture file format.
|
| uuid.pk
| Universally Unique Identifier (UUID) as defined by RFC4122.
|
| redoxfs.pk
| RedoxFS files ystem of Redox OS.
|
| ieee754.pk
| IEEE Standard for Floating-Point Arithmetic.
`----
- The ELF pickle now provides functions implementing ELF hashing.
Build system updates
====================
- It is now supported to configure the poke sources with
--disable-hserver.
Documentation updates
=====================
- Documentation for the `format' language construction has been added
to the poke manual.
Other updates
=============
- A new program poked, for "poke daemon", has been contributed to the
poke distribution by Mohammad-Reza Nabipoor. poked links with
libpoke and uses Unix sockets to act as a broker to communicate with
an instance of a Poke incremental compiler. This is already used by
several user interfaces to poke.
- The machine-interface subsystem has been removed from poke, in favor
of the poked approach.
- The example GUI that was intended to be a test tool for the machine
interface has been removed from the poke distribution.
- Many bugs have been fixed.
--
Jose E. Marchesi
Frankfurt am Main
26 January 2023