Discussion:
Modular arithmetic
George Bakos
2012-09-05 21:39:41 UTC
Permalink
I don't see any discussion regarding adding modular operations to
pcap, i.e. "header[offset:width] % 4 != 0". I've put together a patch
that compiles & honors that (at least on the few systems I've tried),
but was wondering if there were any fundamental reason to avoid it?

Thanks.
g
Guy Harris
2012-09-05 23:19:11 UTC
Permalink
Post by George Bakos
I don't see any discussion regarding adding modular operations to
pcap, i.e. "header[offset:width] % 4 != 0". I've put together a patch
that compiles & honors that (at least on the few systems I've tried),
Does it work if the right-hand side of the % operator isn't a constant power of 2 (e.g., by dividing the LHS by the RHS, multiplying the result by the RHS, and subtracting the result from the LHS)?
George Bakos
2012-09-06 07:36:15 UTC
Permalink
Your recommended approach certainly works:

$ tcpdump -nvr /tmp/DG2-test2 '((ip[2:2] - 20) - (((ip[2:2] - 20) /
5) * 5)) != 0 && ip[6] & 0x20 = 0x20'
reading from file /tmp/DG2-test2, link-type EN10MB (Ethernet)
19:01:51.270202 IP (tos 0x0, ttl 64, id 1, offset 40, flags [+],
proto ICMP (1), length 61) 192.168.11.5 > 192.168.11.46: ip-proto-1

(000) ldh [12]
(001) jeq #0x800 jt 2 jf 16
(002) ldh [16]
(003) sub #20
(004) st M[2]
(005) ldh [16]
(006) sub #20
(007) div #5
(008) mul #5
(009) tax
(010) ld M[2]
(011) jeq x jt 16 jf 12
(012) ldb [20]
(013) and #0x20
(014) jeq #0x20 jt 15 jf 16
(015) ret #65535
(016) ret #0

(obviously optimized)

vs:

$ tcpdump -nvr /tmp/DG2-test2 '(ip[2:2] - 20) % 5 != 0 && ip[6] &
0x20 = 0x20'

reading from file /tmp/DG2-test2, link-type EN10MB (Ethernet)
19:01:51.270202 IP (tos 0x0, ttl 64, id 1, offset 40, flags [+],
proto ICMP (1), length 61) 192.168.11.5 > 192.168.11.46: ip-proto-1

(000) ldh [12]
(001) jeq #0x800 jt 2 jf 10
(002) ldh [16]
(003) sub #20
(004) mod #5
(005) jeq #0x0 jt 10 jf 6
(006) ldb [20]
(007) and #0x20
(008) jeq #0x20 jt 9 jf 10
(009) ret #65535
(010) ret #0

On Wed, 5 Sep 2012 16:19:11 -0700
Post by Guy Harris
Post by George Bakos
I don't see any discussion regarding adding modular operations to
pcap, i.e. "header[offset:width] % 4 != 0". I've put together a patch
that compiles & honors that (at least on the few systems I've tried),
Does it work if the right-hand side of the % operator isn't a constant power of 2 (e.g., by dividing the LHS by the RHS, multiplying the result by the RHS, and subtracting the result from the LHS)?
_______________________________________________
tcpdump-workers mailing list
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
--
Guy Harris
2012-09-06 08:02:32 UTC
Permalink
Post by George Bakos
$ tcpdump -nvr /tmp/DG2-test2 '(ip[2:2] - 20) % 5 != 0 && ip[6] &
0x20 = 0x20'
reading from file /tmp/DG2-test2, link-type EN10MB (Ethernet)
19:01:51.270202 IP (tos 0x0, ttl 64, id 1, offset 40, flags [+],
proto ICMP (1), length 61) 192.168.11.5 > 192.168.11.46: ip-proto-1
(000) ldh [12]
(001) jeq #0x800 jt 2 jf 10
(002) ldh [16]
(003) sub #20
(004) mod #5
(005) jeq #0x0 jt 10 jf 6
OK, so you presumably added a BPF_MOD instruction to the BPF interpreter as part of your changes, right? There's none in libpcap's bpf_filter.c nor in a fairly recent FreeBSD kernel's bpf_filter.c nor in Linux 3.0.4's net/core/filter.c, so that code won't work with at least those interpreters.
George Bakos
2012-09-08 20:31:13 UTC
Permalink
Here's a patch to libpcap-1.3 to test against. I still need to
include changes to man pages.

g

On Sat, 08 Sep 2012 10:03:35 +0200
Gents,
Any fundamental reason why the following (, etc.) shouldn't be
included in net/core/filter.c?
if (X == 0)
return 0;
A %= X;
continue;
Copying netdev.
In principle no reason against it, but you may need to update
the various BPF JITs too that Linux now has too.
Hi Andi, thanks for the forward
In recent commit ffe06c17afbb was added ALU_XOR_X,
so we could add ALU_MOD_X as well.
ALU_MOD_K is a bit more complex as we cant use an ancillary, and must
/* alu/jmp fields */
#define BPF_OP(code) ((code) & 0xf0)
#define BPF_ADD 0x00
#define BPF_SUB 0x10
#define BPF_MUL 0x20
#define BPF_DIV 0x30
#define BPF_OR 0x40
#define BPF_AND 0x50
#define BPF_LSH 0x60
#define BPF_RSH 0x70
#define BPF_NEG 0x80
So I guess we could use
#define BPF_MOD 0x90
We can update them later.
At JIT 'compile' time, if we find a not yet handled instruction, we fall
back to the net/core/filter.c interpreter.
If the following patch is accepted, I'll do the x86 part as a followup.
Thanks !
[PATCH net-next] filter: add MOD operation
Add a new ALU opcode, to compute a modulus.
Commit ffe06c17afbbb used an ancillary to implement XOR_X,
but here we reserve one of the available ALU opcode to implement both
MOD_X and MOD_K
---
include/linux/filter.h | 4 ++++
net/core/filter.c | 15 +++++++++++++++
2 files changed, 19 insertions(+)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 82b0135..3cf5fd5 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -74,6 +74,8 @@ struct sock_fprog { /* Required for SO_ATTACH_FILTER. */
#define BPF_LSH 0x60
#define BPF_RSH 0x70
#define BPF_NEG 0x80
+#define BPF_MOD 0x90
+
#define BPF_JA 0x00
#define BPF_JEQ 0x10
#define BPF_JGT 0x20
@@ -196,6 +198,8 @@ enum {
BPF_S_ALU_MUL_K,
BPF_S_ALU_MUL_X,
BPF_S_ALU_DIV_X,
+ BPF_S_ALU_MOD_K,
+ BPF_S_ALU_MOD_X,
BPF_S_ALU_AND_K,
BPF_S_ALU_AND_X,
BPF_S_ALU_OR_K,
diff --git a/net/core/filter.c b/net/core/filter.c
index 907efd2..fbe3a8d 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -167,6 +167,14 @@ unsigned int sk_run_filter(const struct sk_buff *skb,
A = reciprocal_divide(A, K);
continue;
+ if (X == 0)
+ return 0;
+ A %= X;
+ continue;
+ A %= K;
+ continue;
A &= X;
continue;
@@ -469,6 +477,8 @@ int sk_chk_filter(struct sock_filter *filter, unsigned int flen)
[BPF_ALU|BPF_MUL|BPF_K] = BPF_S_ALU_MUL_K,
[BPF_ALU|BPF_MUL|BPF_X] = BPF_S_ALU_MUL_X,
[BPF_ALU|BPF_DIV|BPF_X] = BPF_S_ALU_DIV_X,
+ [BPF_ALU|BPF_MOD|BPF_K] = BPF_S_ALU_MOD_K,
+ [BPF_ALU|BPF_MOD|BPF_X] = BPF_S_ALU_MOD_X,
[BPF_ALU|BPF_AND|BPF_K] = BPF_S_ALU_AND_K,
[BPF_ALU|BPF_AND|BPF_X] = BPF_S_ALU_AND_X,
[BPF_ALU|BPF_OR|BPF_K] = BPF_S_ALU_OR_K,
@@ -531,6 +541,11 @@ int sk_chk_filter(struct sock_filter *filter, unsigned int flen)
return -EINVAL;
ftest->k = reciprocal_value(ftest->k);
break;
+ /* check for division by zero */
+ if (ftest->k == 0)
+ return -EINVAL;
+ break;
--
David Laight
2012-09-10 08:41:02 UTC
Permalink
Gents,
Any fundamental reason why the following (, etc.) shouldn't be
included in net/core/filter.c?
if (X == 0)
return 0;
A %= X;
continue;
Copying netdev.
In principle no reason against it, but you may need to update
the various BPF JITs too that Linux now has too.
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.

David (***@netbsd.org)
Eric Dumazet
2012-09-10 09:25:24 UTC
Permalink
Post by David Laight
Gents,
Any fundamental reason why the following (, etc.) shouldn't be
included in net/core/filter.c?
if (X == 0)
return 0;
A %= X;
continue;
Copying netdev.
In principle no reason against it, but you may need to update
the various BPF JITs too that Linux now has too.
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.
Yes, does it mean BPF is frozen ?

Or is BSD so hard to update these days ?

modulus can be implemented using fallback to div and sub, I am not sure
libpcap should sense kernel support or not.

George, make sure libpcap optimizer correctly replaces MOD X by AND (X
- 1) if X is a power of two.
David Laight
2012-09-10 10:41:54 UTC
Permalink
Post by Eric Dumazet
Post by David Laight
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.
Yes, does it mean BPF is frozen ?
Or is BSD so hard to update these days ?
Not really - but it some other places that need updating in order
to make this useful for cross-platform tools (like tcpdump).

The 'real fun (tm)' happens when NetBSD tries to run Linux binaries
that include the Linux libpcap.

David
Daniel Borkmann
2012-09-10 11:49:36 UTC
Permalink
Post by David Laight
Post by Eric Dumazet
Post by David Laight
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.
Yes, does it mean BPF is frozen ?
Or is BSD so hard to update these days ?
Not really - but it some other places that need updating in order
to make this useful for cross-platform tools (like tcpdump).
The 'real fun (tm)' happens when NetBSD tries to run Linux binaries
that include the Linux libpcap.
Correct me if I'm wrong, but if you want to run the _Linux_ version of
libpcap, then you would also need PF_PACKET's RX_RING API in NetBSD.
Otherwise, it doesn't run either. Probably then this BPF instruction
is the smaller problem for NetBSD.
Guy Harris
2012-09-10 17:50:30 UTC
Permalink
Post by David Laight
Post by Eric Dumazet
Post by David Laight
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.
Yes, does it mean BPF is frozen ?
Or is BSD so hard to update these days ?
Not really - but it some other places that need updating in order
to make this useful for cross-platform tools (like tcpdump).
The 'real fun (tm)' happens when NetBSD tries to run Linux binaries
that include the Linux libpcap.
Additional fun happens when a Linux system with a kernel that doesn't have the mod instruction tries to run Linux binaries that include a Linux libpcap that generates code using the mod instruction; this is not a BSD-vs.-Linux issue, it's a "kernel that lacks the mod instruction vs. libpcap that generates code that includes the mod instruction" issue.

BSD/OS X, Linux, Solaris, and the WinPcap driver need, if they adopt new BPF instructions, to have a mechanism by which libpcap (or anything else using BPF filtering) can inquire about the capabilities of the OS BPF interpreter; libpcap would use that to determine what code to generate if generating code for the in-kernel BPF interpreter.

(Please reply to tcpdump-***@lists.tcpdump.org as well as to ***@vger.kernel.org, so that people not on both of those lists can follow the discussion. Messages from non-members of tcpdump-workers to tcpdump-workers shouldn't bounce, but they do need to be approved by a moderator; Michael, if you want to at least temporarily turn the flood on for my e-mail address, so I can help moderate, go ahead.)
Guy Harris
2014-05-18 18:26:46 UTC
Permalink
Post by Guy Harris
Post by David Laight
Post by Eric Dumazet
Post by David Laight
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.
Yes, does it mean BPF is frozen ?
Or is BSD so hard to update these days ?
Not really - but it some other places that need updating in order
to make this useful for cross-platform tools (like tcpdump).
The 'real fun (tm)' happens when NetBSD tries to run Linux binaries
that include the Linux libpcap.
Additional fun happens when a Linux system with a kernel that doesn't have the mod instruction tries to run Linux binaries that include a Linux libpcap that generates code using the mod instruction; this is not a BSD-vs.-Linux issue, it's a "kernel that lacks the mod instruction vs. libpcap that generates code that includes the mod instruction" issue.
BSD/OS X, Linux, Solaris, and the WinPcap driver need, if they adopt new BPF instructions, to have a mechanism by which libpcap (or anything else using BPF filtering) can inquire about the capabilities of the OS BPF interpreter; libpcap would use that to determine what code to generate if generating code for the in-kernel BPF interpreter.
In this particular case, there's no choice about what to generate - either you have mod and XOR instructions or you don't.

Furthermore, if the kernel rejects a particular BPF program, libpcap just falls back on filtering in userland, so we can support % and ^ operators in libpcap on all platforms, albeit with a performance hit on Linux prior to 3.7 and, currently, on other platforms. I don't think there's anything inherently troublesome about BPF_MOD and BPF_XOR, so I'll file bugs against the various BSDs and OS X suggesting that they be added.

So I've checked in a change to libpcap to implement BPF_MOD and BPF_XOR and to add % and ^ operators (yes, the same zero-divide checks in bpf_validate() done for BPF_DIV are done for BPF_MOD, and the optimizer was updated to handle BPF_MOD like BPF_DIV and to handle BPF_XOR like BPF_OR). The pcap-filter man page documents it, but warns that filters using them might be run in userland on pre-3.7 Linux and non-Linux platforms.
Andi Kleen
2012-09-10 14:49:28 UTC
Permalink
Post by David Laight
What about the other OS - eg all the BSDs?
I had a vague idea that BPF was supposed to be reasonable portable.
Linux already has a variety of BPF extensions I believe.
But most of them were not targetted for tcpdump, but for other tools.

-Andi
--
***@linux.intel.com -- Speaking for myself only
Continue reading on narkive:
Loading...