Printing PPI packets

[Darren Reed]

Printing PPI packets with tcpdump does not turn out
to be that hard.

My simple tests have produced the output as below.

It would be worthwhile having some changes made into
the tcpdump code base that were similar to the attached
that print them out.

Darren

19:20:51.470264 , DLT IPV4 (228) len 0, length 76: ip: (tos 0x0, ttl 255, id 509, offset 0, flags [+, DF], proto ICMP (1), length 68)
1.1.1.1 > 1.1.1.2: ICMP echo request, id 35462, seq 3, length 48
0x0000: 0000 0000 0000 00e4 4500 0044 01fd 6000
0x0010: ff01 55b7 0101 0101 0101 0102 0800 45bd
0x0020: 8a86 0003 4d9f c283 0007 2c8c 0809 0a0b
0x0030: 0c0d 0e0f 1011 1213 1415 1617 1819 1a1b
0x0040: 1c1d 1e1f 2021 2223 2425 2627
19:20:52.470238 , DLT IPV4 (228) len 0, length 76: ip: (tos 0x0, ttl 255, id 510, offset 0, flags [+, DF], proto ICMP (1), length 68)
1.1.1.1 > 1.1.1.2: ICMP echo request, id 35462, seq 4, length 48
0x0000: 0000 0000 0000 00e4 4500 0044 01fe 6000
0x0010: ff01 55b6 0101 0101 0101 0102 0800 45cc
0x0020: 8a86 0004 4d9f c284 0007 2c7b 0809 0a0b
0x0030: 0c0d 0e0f 1011 1213 1415 1617 1819 1a1b
0x0040: 1c1d 1e1f 2021 2223 2425 2627

[Michael Richardson]

Darren> Printing PPI packets with tcpdump does not turn out to be
Darren> that hard.

Darren> My simple tests have produced the output as below.

Super!
Do you have some PPI pcap files we can include in the test cases?

I've committed your code, will git push when I get online.

[Darren Reed]

Michael,

See the attached file for a sample of IPv4 packets captured.

There are also libpcap issues here that need to be resolved. At present,
using any filter with a PPI device fails to match any packet that
doesn't have a DLT of DLT_IEEE802_11.

Darren

Darren> Printing PPI packets with tcpdump does not turn out to be
Darren> that hard.
Darren> My simple tests have produced the output as below.
Super!
Do you have some PPI pcap files we can include in the test cases?
I've committed your code, will git push when I get online.

[Guy Harris]

There are also libpcap issues here that need to be resolved. At present, using any filter with a PPI device fails to match any packet that doesn't have a DLT of DLT_IEEE802_11.

...which is one of the things wrong with PPI. pcap-ng makes the link-layer type part of the record header, rather than part of the record data, so you don't have to special-case the filtering routine (so that, for PPI, the filter routine would pull the link-layer type out and run different filter programs based on the link-layer type) or have the BPF compiler generate, for PPI, code that switches off the link-layer type and includes code to do the filtering for all possible link-layer types.

pcap-ng, and the first of those strategies for PPI, would require a routine that sets the filter on a pcap_t to a string, so that code can be generated while reading the savefile as each link-layer type is discovered (unless you generate code for every possible link-layer type and stash it away "just in case", but even that requires a different API).-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[Darren Reed]

Post by Guy Harris

There are also libpcap issues here that need to be resolved. At present, using any filter with a PPI device fails to match any packet that doesn't have a DLT of DLT_IEEE802_11.

...which is one of the things wrong with PPI. pcap-ng makes the link-layer type part of the record header, rather than part of the record data, so you don't have to special-case the filtering routine (so that, for PPI, the filter routine would pull the link-layer type out and run different filter programs based on the link-layer type) or have the BPF compiler generate, for PPI, code that switches off the link-layer type and includes code to do the filtering for all possible link-layer types.
pcap-ng, and the first of those strategies for PPI, would require a routine that sets the filter on a pcap_t to a string, so that code can be generated while reading the savefile as each link-layer type is discovered (unless you generate code for every possible link-layer type and stash it away "just in case", but even that requires a different API).

I see - you're concerned about how do you make "tcpdump icmp" work when
the link type is PPI (or pcap-ng) and the contents of the file could be
a mixture of ethernet, Infiniband and others. The problem being that the
location of the IP header is different in each, requiring a different
program to be generated and applied.

Darren

[Guy Harris]

I see - you're concerned about how do you make "tcpdump icmp" work when the link type is PPI (or pcap-ng)

Presumably meaning "when the link type is PPI or when the file is a pcap-ng file" (pcap-ng isn't a link type, it's a file format).

and the contents of the file could be a mixture of ethernet, Infiniband and others. The problem being that the location of the IP header is different in each, requiring a different program to be generated and applied.

The location of the IP header *and* the location and values of the type field in the link-layer header are different.-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[Darren Reed]

Post by Guy Harris

I see - you're concerned about how do you make "tcpdump icmp" work when the link type is PPI (or pcap-ng)

Presumably meaning "when the link type is PPI or when the file is a pcap-ng file" (pcap-ng isn't a link type, it's a file format).

and the contents of the file could be a mixture of ethernet, Infiniband and others. The problem being that the location of the IP header is different in each, requiring a different program to be generated and applied.

The location of the IP header *and* the location and values of the type field in the link-layer header are different.

In terms of pcap, I'm becoming more and more of the opinion that DLT_PPI
should not be used for anything other than DLT_IEEE802_11.

The reason being is that it increases the complexity of generating BPF
far in excess of its worth to try and handle the "special case" of
"length == 0" along with the current use of DLT_PPI.

Whilst this may actually serve the purposes of the file format and
printing packets, in terms of a solution for filtering packets of varied
data link types, it just doesn't work.

In terms of pcap, I'd thus like to suggest that we add a DLT_LINK that
has a header like this:

version=0 (8 bits)
reserved=0 (8 bits)
dlt_hdr_len (16 bits)
SAP (32 bits)
DLT (32 bits)

The dlt_hdr_len is the length of the ethernet header or the 802.11
header or the infiniband header or...

The value of the SAP has meaning only for a specific value of the DLT.
Whilst some DLTs may use the same set of values for the SAP field, this
is not guaranteed.

There's no length field because the pcap header already contains the length

For example, an ethernet packet might have SAP=0x0800 (ETHERTYPE_IP) and
DLT=0x1 (DLT_EN10MB)


Why am I not very interested in pcap-ng?
"The pcapng file format specification is still work in progress, see:"

... moving target.

Darren

[Guy Harris]

In terms of pcap, I'm becoming more and more of the opinion that DLT_PPI should not be used for anything other than DLT_IEEE802_11.

Sounds good to me.

Why am I not very interested in pcap-ng?
"The pcapng file format specification is still work in progress, see:"
... moving target.

pcap-ng is extensible, so it's *always* going to be a moving target. However, both Wireshark and libpcap read it, so there are limits on how much it can move; the spec needs to be clarified to indicate what part isn't going to change and what part is.-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[Darren Reed]

Post by Guy Harris

In terms of pcap, I'm becoming more and more of the opinion that DLT_PPI should not be used for anything other than DLT_IEEE802_11.

Sounds good to me.

Why am I not very interested in pcap-ng?
"The pcapng file format specification is still work in progress, see:"
... moving target.

pcap-ng is extensible, so it's *always* going to be a moving target. However, both Wireshark and libpcap read it, so there are limits on how much it can move; the spec needs to be clarified to indicate what part isn't going to change and what part is.

Looking through it, the first observation I'd make is that there should
not have been any 16 bit fields. The one that concerns me most is the
IDB which has a 16bit link type.

On Solaris, we're using numbers above 0x80000000 to represent a
"private" number space for DLPI link types that correspond to internal
features. I can see that it being desirable to continue to be able to do
that, even if only for the purposes of experimentation. At present,
those DLPI numbers typically relate to a more "normal" link type, such
as DL_IPV4 (0x80000001) being LINKTYPE_IPV4 but others may not.

Darren

[Guy Harris]

Looking through it, the first observation I'd make is that there should not have been any 16 bit fields. The one that concerns me most is the IDB which has a 16bit link type.

We could add an "enhanced IDB" with a 32-bit LinkType field.

On Solaris, we're using numbers above 0x80000000 to represent a "private" number space for DLPI link types that correspond to internal features. I can see that it being desirable to continue to be able to do that, even if only for the purposes of experimentation. At present, those DLPI numbers typically relate to a more "normal" link type, such as DL_IPV4 (0x80000001) being LINKTYPE_IPV4 but others may not.

If a link-layer type is desired for internal experimentation within an organization, with the organization having no reason to care whether those captures will be properly interpreted outside the organization, you can use one of the LINKTYPE_USERn types.

If you want the files to be usable outside the organization - i.e., if you want a vendor namespace for link-layer types (which means it applies to pcap as well as pcap-ng) - we'd need to structure the link-layer type values to include an organization ID. I've already put some work into that to support some NetBSD-specific link-layer types; it never went any further, but it splits the link-layer type value into:

the upper 4 bits, for an FCS length field, plus the next bit whose purpose I'd have to remember, plus the next bit, which indicates whether the FCS length is present or not;

the next 10 bits, for a "class" value that would be an organization ID;

the next 16 bits, for link-layer type values within that class.

-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[Guy Harris]

Post by Guy Harris

Looking through it, the first observation I'd make is that there should not have been any 16 bit fields. The one that concerns me most is the IDB which has a 16bit link type.

We could add an "enhanced IDB" with a 32-bit LinkType field.

...and the only remaining 16-bit fields are:

the major and minor version numbers in the Section Header Block - yeah, I know, 640K and all that, but I doubt that'll be an issue, especially for the major version number (if you completely incompatibly change the file format more than 65535 times in the next 1000 years, UR DOING IT WRONG, and even for the minor version number, given the extensibility of the format, I don't see much need for version number changes);

the Interface ID and Drops Count in the Packet Block, but that's been deprecated in favor of the Extended Packet Block, with a 32-bit interface ID and a 64-bit drop count as an option;

the Record Type and Record Length in the Name Resolution Block - I'm not sure there will be more than 65533 more name types, so the Record Type is probably OK at 16 bits, but I guess you could either have really really long network addresses or, more likely, a huge number of names corresponding to an address, so the 16-bit Record Length might be an issue, so there might have to be an Extended Name Resolution Block.-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[Darren Reed]

Post by Guy Harris

Post by Guy Harris

Looking through it, the first observation I'd make is that there should not have been any 16 bit fields. The one that concerns me most is the IDB which has a 16bit link type.

We could add an "enhanced IDB" with a 32-bit LinkType field.

the major and minor version numbers in the Section Header Block - yeah, I know, 640K and all that, but I doubt that'll be an issue, especially for the major version number (if you completely incompatibly change the file format more than 65535 times in the next 1000 years, UR DOING IT WRONG, and even for the minor version number, given the extensibility of the format, I don't see much need for version number changes);
the Interface ID and Drops Count in the Packet Block, but that's been deprecated in favor of the Extended Packet Block, with a 32-bit interface ID and a 64-bit drop count as an option;
the Record Type and Record Length in the Name Resolution Block - I'm not sure there will be more than 65533 more name types, so the Record Type is probably OK at 16 bits, but I guess you could either have really really long network addresses or, more likely, a huge number of names corresponding to an address, so the 16-bit Record Length might be an issue, so there might have to be an Extended Name Resolution Block.

I looked over the other 16 bit fields and came to the same conclusion as
yourself about the above so didn't mention them.

In the breakup where you were suggesting 10 bits that could be an
organization ID, reserve "0" for the publicly recognised set and all 1's
for private/experimental. Is 10 bits enough for an organisation ID? How
many organisations have received a number for use with SNMP? I'm not
saying that as many organisations will need an identifier here but
rather that the number of organisations might be larger than you would
suspect at first.

Darren

[Guy Harris]

In the breakup where you were suggesting 10 bits that could be an organization ID, reserve "0" for the publicly recognised set

That's already done (implicitly, by virtue of those bits being 0 in existing LINKTYPE_ values, and explicitly as well).

and all 1's for private/experimental.

Sounds good, although it is somewhat redundant with LINKTYPE_USERn (presumably "private/experimental" means "multiple organizations can use the same value for different purposes, so don't request that the tcpdump/Wireshark/etc. developers put into their official releases anything that assigns a particular interpretation to any of those values").

Is 10 bits enough for an organisation ID? How many organisations have received a number for use with SNMP? I'm not saying that as many organisations will need an identifier here but rather that the number of organisations might be larger than you would suspect at first.

Well, given that the other bits I took aren't necessary for pcap-ng, we could use the reserved field in the existing IDB for an organization code, and, if and when we get above 1023 organizations, say "sorry, usable only in pcap-ng". If by "how many organisations have received a number for use with SNMP?" you mean "how many organizations have SMI Network Management Private Enterprise Codes?", the answer is "probably somewhere around 37,891, modulo holes in the number space" as of today:

http://www.iana.org/assignments/enterprise-numbers

Using IEEE OUI's would require 24 bits, leaving only 8 bits for existing LINKTYPE_ values, and we're already up to 238, so that's not acceptable. Besides, OUIs cost a minimum of USD 1,750:

http://standards.ieee.org/develop/regauth/oui/index.html

which is another reason not to use them.-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.

[Guy Harris]

Post by Darren Reed
Printing PPI packets with tcpdump does not turn out
to be that hard.
My simple tests have produced the output as below.

Your simple tests were with invalid PPI files; as the PPI spec:

http://www.cacetech.com/documents/PPI%20Header%20format%201.0.7.pdf

says:

Multi-byte integers in the packet header and field headers MUST be stored as little-endian. The endianness of field data may be either big- or little-endian, and MUST be noted in the field description. The total length of the packet header plus all field headers and field data MUST be padded to a 32-bit boundary.

and the code does

len = EXTRACT_16BITS(&hdr->ppi_len);
dlt = EXTRACT_32BITS(&hdr->ppi_dlt);

which treats the fields in the packet header as big-endian, not little-endian, so...

Post by Darren Reed
19:20:51.470264 , DLT IPV4 (228) len 0, length 76: ip: (tos 0x0, ttl 255, id 509, offset 0, flags [+, DF], proto ICMP (1), length 68)
1.1.1.1 > 1.1.1.2: ICMP echo request, id 35462, seq 3, length 48
0x0000: 0000 0000 0000 00e4 4500 0044 01fd 6000
0x0010: ff01 55b7 0101 0101 0101 0102 0800 45bd
0x0020: 8a86 0003 4d9f c283 0007 2c8c 0809 0a0b
0x0030: 0c0d 0e0f 1011 1213 1415 1617 1819 1a1b
0x0040: 1c1d 1e1f 2021 2223 2425 2627

...if that packet really had 0x00 0xE4 rather than 0xE4 0x00 in the DLT field, that packet has a DLT value of 58368, which is not a currently-assigned LINKTYPE_/DLT_ value.

I've checked a fix in to treat the length and DLT values as little-endian.