{"id":1961,"date":"2016-02-03T06:46:08","date_gmt":"2016-02-03T14:46:08","guid":{"rendered":"https:\/\/www.privateinternetaccess.com\/blog\/?p=1961"},"modified":"2024-02-01T02:16:19","modified_gmt":"2024-02-01T10:16:19","slug":"linux-networking-stack-from-the-ground-up-part-4-2","status":"publish","type":"post","link":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/","title":{"rendered":"Linux networking stack from the ground up, part 5"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-1\/\">part 1<\/a> | <a href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-2\/\">part 2<\/a> | <a href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-3\/\">part 3<\/a> | <a href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4\/\">part 4<\/a> | <a href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\">part 5<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Overview<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This blog post picks up right where part 4 left off and begins by examining the last part of the IP protocol stack, the handoff to the UDP protocol stack, and finally by queuing the data to a socket\u2019s queue so it can be read by user programs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><code>ip_rcv_finish<\/code><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Once net filter has had a chance to take a look at the packet and decide what to do with it, <code>ip_rcv_finish<\/code> is called.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>ip_rcv_finish<\/code> begins with an optimization. In order to deliver the packet to proper place, a <code>dst_entry<\/code> from the routing system needs to in place. In order to obtain one, the code initially attempts to call the <code>early_demux<\/code> function from the higher level protocol.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>early_demux<\/code> routine is an optimization which attempts to find the <code>dst_entry<\/code> needed to deliver the packet by checking if a <code>dst_entry<\/code> is cached on the socket.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here\u2019s what that looks like (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/ip_input.c#L317-L327\">net\/ipv4\/ip_input.c:317<\/a>):<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">if (sysctl_ip_early_demux &amp;&amp; !skb_dst(skb) &amp;&amp; skb-&gt;sk == NULL) {\n  const struct net_protocol *ipprot;\n  int protocol = iph-&gt;protocol;\n\n  ipprot = rcu_dereference(inet_protos[protocol]);\n  if (ipprot &amp;&amp; ipprot-&gt;early_demux) {\n    ipprot-&gt;early_demux(skb);\n    \/* must reload iph, skb-&gt;head might have changed *\/\n    iph = ip_hdr(skb);\n  }\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If the optimization is disabled or there is no cached entry (because this is the first UDP packet arriving), the packet will be handed off to the routing system in the kernel where the <code>dst_entry<\/code> will be computed and assigned.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once the routing layer completes, statistics counters are updated and the function ends by calling <code>dst_input(skb)<\/code> which in turn calls the <code>input<\/code> function pointer on the packet\u2019s <code>dst_entry<\/code> structure that was affixed by the routing system.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the packet\u2019s final destination is the local system, the routing system will attach the function <code>ip_local_deliver<\/code> to the <code>input<\/code> function pointer in the <code>dst_entry<\/code> structure on the packet.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><code>ip_local_deliver<\/code> and netfilter<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Recall how we saw the following pattern in the IP protocol layer:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Calls to <code>ip_rcv<\/code> do some initial bookkeeping.<\/li><li>Packet is handed off to netfilter for processing, with a pointer to a callback to be executed when processing finishes.<\/li><li><code>ip_rcv_finish<\/code> is the callback which finished processing and continued working toward pushing the packet up the networking stack<\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\"><code>ip_local_deliver<\/code> has the same pattern (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/ip_input.c#L242-L258\">net\/ipv4\/ip_input.c:242<\/a>):<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/*      \n *      Deliver IP Packets to the higher protocol layers.                                  \n *\/             \nint ip_local_deliver(struct sk_buff *skb)\n{       \n  \/*                                                                                 \n   *  Reassemble IP fragments.                                                   \n   *\/\n\n  if (ip_is_fragment(ip_hdr(skb))) {\n    if (ip_defrag(skb, IP_DEFRAG_LOCAL_DELIVER))                               \n      return 0;\n  }\n\n  return NF_HOOK(NFPROTO_IPV4, NF_INET_LOCAL_IN, skb, skb-&gt;dev, NULL, ip_local_deliver_finish);                                           \n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Except that in this case, the netfilter chain is <code>NF_INET_LOCAL_IN<\/code> and the <code>okfn<\/code> to be called on completion is <code>ip_local_deliver_finish<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We examined how packets are moved through netfilter briefly earlier, so we\u2019ll move on to the completion <code>ip_local_deliver_finish<\/code>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><code>ip_local_deliver_finish<\/code><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><code>ip_local_deliver_finish<\/code> obtains the protocol from the packet, looks up a <code>net_protocol<\/code> structure registered for that protocol, and calls the function pointed to by <code>handler<\/code> in the <code>net_protocol<\/code> structure.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This hands the packet up to the higher level protocol layer.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Higher level protocol registration<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In our case, we care mostly about UDP, but TCP protocol handlers are registered the same way and at the same time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">On <a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/af_inet.c#L1526-L1547\">net\/ipv4\/af_inet.c:1553<\/a> we can find the structure definitions which contains the handler functions for connecting the UDP, TCP, and ICMP protocols to the IP protocol layer:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">static const struct net_protocol tcp_protocol = {\n  .early_demux    =       tcp_v4_early_demux,\n  .handler        =       tcp_v4_rcv,\n  .err_handler    =       tcp_v4_err,\n  .no_policy      =       1,\n  .netns_ok       =       1,\n};\n\nstatic const struct net_protocol udp_protocol = {\n  .early_demux =  udp_v4_early_demux,\n  .handler =      udp_rcv,\n  .err_handler =  udp_err,\n  .no_policy =    1,\n  .netns_ok =     1,\n};\n\nstatic const struct net_protocol icmp_protocol = {\n  .handler =      icmp_rcv,\n  .err_handler =  icmp_err,\n  .no_policy =    1,\n  .netns_ok =     1,\n};<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">These structures are registered in the initialization code of the inet address family (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/af_inet.c#L1716-L1725\">net\/ipv4\/af_inet.c:1716<\/a>):<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/*\n  *      Add all the base protocols.\n  *\/\n\n if (inet_add_protocol(&amp;icmp_protocol, IPPROTO_ICMP) &lt; 0)\n    pr_crit(\"%s: Cannot add ICMP protocol\\n\", __func__);\n if (inet_add_protocol(&amp;udp_protocol, IPPROTO_UDP) &lt; 0)\n    pr_crit(\"%s: Cannot add UDP protocol\\n\", __func__);\n if (inet_add_protocol(&amp;tcp_protocol, IPPROTO_TCP) &lt; 0)\n    pr_crit(\"%s: Cannot add TCP protocol\\n\", __func__);<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In our research case, we care mostly about UDP. So, we\u2019ll examine the UDP <code>handler<\/code> function which is called from <code>ip_local_deliver_finish<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As we see in the structure definition above, this function is called <code>udp_rcv<\/code>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">UDP<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The code for the UDP protocol layer can be found in: <a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/udp.c\">net\/ipv4\/udp.c<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">udp_rcv<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>udp_rcv<\/code> (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/udp.c#L1954\">net\/ipv4\/udp.c:1954<\/a>) function is just a single line which calls directly into <code>__udp4_lib_rcv<\/code> to handle receiving the packet.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><code>__udp4_lib_rcv<\/code><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>__udp4_lib_rcv<\/code> (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/udp.c#L1704-L1820\">net\/ipv4\/udp.c:1708<\/a>) will check to ensure the packet is valid and obtain the UDP header, UDP datagram length, source address, and destination address. Next, are some additional integrity checks and checksum verification.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Recall that earlier in the IP protocol layer, we saw that an optimization is performed to attach a <code>dst_entry<\/code> to the packet before it is handed off to the upper layer protocol (UDP in our case).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If a socket and corresponding <code>dst_entry<\/code> were found, <code>__udp4_lib_rcv<\/code> will queue the packet to be received by the socket:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sk = skb_steal_sock(skb);                                                          \nif (sk) {\n  struct dst_entry *dst = skb_dst(skb);                                      \n  int ret;                                                                   \n                                                                                   \n  if (unlikely(sk-&gt;sk_rx_dst != dst))                                        \n    udp_sk_rx_dst_set(sk, dst);                                        \n                                                                                   \n  ret = udp_queue_rcv_skb(sk, skb);                                          \n  sock_put(sk);                                                              \n  \/* a return value &gt; 0 means to resubmit the input, but                     \n   * it wants the return to be -protocol, or 0                               \n   *\/                                                                        \n  if (ret &gt; 0)                                                               \n    return -ret;                                                       \n  return 0;                                                                  \n} else {<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If there is no socket attached from the <code>early_demux<\/code> operation, a receiving socket will now be looked up by calling <code>__udp4_lib_lookup_skb<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In both cases described above, the datagram will be queued to the socket:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">ret = udp_queue_rcv_skb(sk, skb);\nsock_put(sk);<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If no socket was found, the datagram will be dropped:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/* No socket. Drop packet silently, if checksum is wrong *\/\nif (udp_lib_checksum_complete(skb))\n  goto csum_error;\n\nUDP_INC_STATS_BH(net, UDP_MIB_NOPORTS, proto == IPPROTO_UDPLITE);\nicmp_send(skb, ICMP_DEST_UNREACH, ICMP_PORT_UNREACH, 0);\n\n\/*\n * Hmm.  We got an UDP packet to a port to which we\n * don't wanna listen.  Ignore it.\n *\/\nkfree_skb(skb);\nreturn 0;<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">udp_queue_rcv_skb<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The initial parts of this function are as follows:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Determine if the socket associated with the datagram is an encapsulation socket. If so, pass the packet up to that layer\u2019s handler function before proceeding.<\/li><li>Determine if the datagram is a UDP-Lite datagram and do some integrity checks.<\/li><li>Verify the UDP checksum of the datagram and drop it if the checksum fails.<\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Finally, we arrive at the receive queue logic (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/udp.c#L1548\">net\/ipv4\/udp.c:1548<\/a>) which begins by checking if the receive queue for the socket is full:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">if (sk_rcvqueues_full(sk, skb, sk-&gt;sk_rcvbuf))                                     \n  goto drop;<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><code>sk_rcvqueues_full<\/code> and tuning receive queue memory<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>sk_rcvqueues_full<\/code> function (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/include\/net\/sock.h#L788-L799\">include\/net\/sock.h:788<\/a>) checks the socket\u2019s backlog length and the socket\u2019s <code>sk_rmem_alloc<\/code> to determine if the sum is greater than the <code>sk_rcvbuf<\/code> for the socket (<code>sk-&gt;sk_rcvbuf<\/code> above):<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\/*                                                                                         \n * Take into account size of receive queue and backlog queue                               \n * Do not take into account this skb truesize,\n * to allow even a single big packet to come.                                              \n *\/                                                                                        \nstatic inline bool sk_rcvqueues_full(const struct sock *sk, const struct sk_buff *skb,     \n                                     unsigned int limit)                                   \n{       \n  unsigned int qsize = sk-&gt;sk_backlog.len + atomic_read(&amp;sk-&gt;sk_rmem_alloc);         \n        \n  return qsize &gt; limit;\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Tuning these values is a bit tricky as there are many things that can be adjusted.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>sk-&gt;sk_rcvbuf<\/code> (called <code>limit<\/code> in the function above) value can be increased to the<br><code>net.core.rmem_max<\/code>. You can set that max by setting the <code>sysctl<\/code>: <code>sysctl -w net.core.rmem_max=8388608<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>sk-&gt;sk_rcvbuf<\/code> starts at the <code>net.core.rmem_default<\/code> value, which can also be adjusted by setting the <code>sysctl<\/code>: <code>sysctl -w net.core.rmem_default=8388608<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You can also set the <code>sk-&gt;sk_rcvbuf<\/code> size by calling <code>setsockopt<\/code> and passing <code>SO_RCVBUF<\/code>. The maximum you can set with <code>setsockopt<\/code> is <code>net.core.rmem_max<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You can override the <code>SO_RCVBUF<\/code> limit by calling <code>setsockopt<\/code> and passing <code>SO_RCVBUFFORCE<\/code>, but the user running the application will need the <code>CAP_NET_ADMIN<\/code> capability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>sk-&gt;sk_rmem_alloc<\/code> value is incremented by calls to <code>skb_set_owner_r<\/code> which set the owner socket of a datagram. We\u2019ll see this called later in the UDP layer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>sk-&gt;sk_backlog.len<\/code> is incremented by calls to <code>sk_add_backlog<\/code>, which we\u2019ll see next.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Back to <code>udp_queue_rcv_skb<\/code><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Once we\u2019ve verified that the queue is not full, we can continue toward queuing the datagram:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">bh_lock_sock(sk);                                                                  \nif (!sock_owned_by_user(sk))                                                       \n  rc = __udp_queue_rcv_skb(sk, skb);                                         \nelse if (sk_add_backlog(sk, skb, sk-&gt;sk_rcvbuf)) {                                 \n  bh_unlock_sock(sk);                                                        \n  goto drop;                                                                 \n}                                                                                  \nbh_unlock_sock(sk);                                                                \n                                                                                   \nreturn rc;<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The first step is determine if the socket currently has any system calls against it from a userland program. If it does not, the datagram can be added to the receive queue with a call to <code>__udp_queue_rcv_skb<\/code>. If it does, the datagram is queued to the backlog.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The datagrams on the backlog are added to the receive queue when socket system calls release the sock with a call to <code>release_sock<\/code>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><code>__udp_queue_rcv_skb<\/code><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>__udp_queue_rcv_skb<\/code> (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/ipv4\/udp.c#L1422\">net\/ipv4\/udp.c:1422<\/a>) function adds datagrams to the receive queue and bumps statistics counters if the datagram could not be added to the receive queue for the socket:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">rc = sock_queue_rcv_skb(sk, skb);                                                  \nif (rc &lt; 0) {                                                                      \n  int is_udplite = IS_UDPLITE(sk);                                           \n                                                                                   \n  \/* Note that an ENOMEM error is charged twice *\/                           \n  if (rc == -ENOMEM)                                                         \n    UDP_INC_STATS_BH(sock_net(sk), UDP_MIB_RCVBUFERRORS, is_udplite);                                      \n    UDP_INC_STATS_BH(sock_net(sk), UDP_MIB_INERRORS, is_udplite);              \n    kfree_skb(skb);                                                            \n    trace_udp_fail_queue_rcv_skb(rc, sk);                                      \n    return -1;                                                                 \n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">To add the datagram to the queue, <code>sock_queue_rcv_skb<\/code> is called.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><code>sock_queue_rcv_skb<\/code><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><code>sock_queue_rcv<\/code> (<a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/v3.13\/net\/core\/sock.c#L388\">net\/core\/sock.c:388<\/a>) does a few things before adding the datagram to the queue:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>The socket\u2019s allocated memory is checked to determine if it has exceeded the receive buffer size. If so, the drop count for the socket is incremented.<\/li><li>Next <code>sk_filter<\/code> is used to process any Berkeley Packet Filter filters that have been applied to the socket.<\/li><li><code>sk_rmem_schedule<\/code> is run to ensure sufficient receive buffer space exists to accept this datagram.<\/li><li>Next the size of the datagram is charged to the socket with a call to <code>skb_set_owner_r<\/code>. This increments <code>sk-&gt;sk_rmem_alloc<\/code>.<\/li><li>The data is added to the queue with a call to <code>__skb_queue_tail<\/code><\/li><li>Finally, any processes waiting on data to arrive in the socket are notified with a call to the <code>sk_data_ready<\/code> notification handler function.<\/li><\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">The End<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">That is how data that arrives from the network ends up on the receive queue for a socket ready to be read by a user process.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP protocol stack, the handoff to the UDP protocol stack, and finally by queuing the data to a socket\u2019s queue &hellip; <a href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Linux networking stack from the ground up, part 5&#8221;<\/span><\/a><\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_stopmodifiedupdate":false,"_modified_date":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-1961","post","type-post","status-publish","format-standard","hentry","category-news"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v26.9) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Linux networking stack from the ground up, part 5<\/title>\n<meta name=\"description\" content=\"part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Linux networking stack from the ground up, part 5\" \/>\n<meta property=\"og:description\" content=\"part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\" \/>\n<meta property=\"og:site_name\" content=\"PIA\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/privateinternetaccess\/\" \/>\n<meta property=\"article:published_time\" content=\"2016-02-03T14:46:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-01T10:16:19+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.privateinternetaccess.com\/blog\/wp-content\/uploads\/2018\/07\/ogimage.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"PIA Research\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@buyvpnservice\" \/>\n<meta name=\"twitter:site\" content=\"@buyvpnservice\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"PIA Research\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\"},\"author\":{\"name\":\"PIA Research\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/person\/867d81a36eafaf83e91f6528aca0ba29\"},\"headline\":\"Linux networking stack from the ground up, part 5\",\"datePublished\":\"2016-02-03T14:46:08+00:00\",\"dateModified\":\"2024-02-01T10:16:19+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\"},\"wordCount\":1188,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#organization\"},\"articleSection\":[\"General Privacy News\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\",\"url\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\",\"name\":\"Linux networking stack from the ground up, part 5\",\"isPartOf\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#website\"},\"datePublished\":\"2016-02-03T14:46:08+00:00\",\"dateModified\":\"2024-02-01T10:16:19+00:00\",\"description\":\"part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP\",\"breadcrumb\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.privateinternetaccess.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Linux networking stack from the ground up, part 5\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#website\",\"url\":\"https:\/\/www.privateinternetaccess.com\/blog\/\",\"name\":\"PIA\",\"description\":\"Online privacy news from around the world.\",\"publisher\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.privateinternetaccess.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#organization\",\"name\":\"Private Internet Access\",\"url\":\"https:\/\/www.privateinternetaccess.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.privateinternetaccess.com\/blog\/wp-content\/uploads\/2018\/07\/pialogowhitekglogo.png\",\"contentUrl\":\"https:\/\/www.privateinternetaccess.com\/blog\/wp-content\/uploads\/2018\/07\/pialogowhitekglogo.png\",\"width\":1200,\"height\":1200,\"caption\":\"Private Internet Access\"},\"image\":{\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/privateinternetaccess\/\",\"https:\/\/x.com\/buyvpnservice\",\"https:\/\/www.instagram.com\/piavpn\/\",\"https:\/\/www.youtube.com\/channel\/UClyJZ47Rizb1xnwuKXDI0_w\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/person\/867d81a36eafaf83e91f6528aca0ba29\",\"name\":\"PIA Research\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/717a02042be28ce22f2ae82923cdaa82551205b8768e3cd4a8fce14988ed0ccd?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/717a02042be28ce22f2ae82923cdaa82551205b8768e3cd4a8fce14988ed0ccd?s=96&d=mm&r=g\",\"caption\":\"PIA Research\"},\"sameAs\":[\"https:\/\/www.privateinternetaccess.com\"],\"url\":\"https:\/\/www.privateinternetaccess.com\/blog\/author\/piaresearch\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Linux networking stack from the ground up, part 5","description":"part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/","og_locale":"en_US","og_type":"article","og_title":"Linux networking stack from the ground up, part 5","og_description":"part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP","og_url":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/","og_site_name":"PIA","article_publisher":"https:\/\/www.facebook.com\/privateinternetaccess\/","article_published_time":"2016-02-03T14:46:08+00:00","article_modified_time":"2024-02-01T10:16:19+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/www.privateinternetaccess.com\/blog\/wp-content\/uploads\/2018\/07\/ogimage.png","type":"image\/png"}],"author":"PIA Research","twitter_card":"summary_large_image","twitter_creator":"@buyvpnservice","twitter_site":"@buyvpnservice","twitter_misc":{"Written by":"PIA Research","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#article","isPartOf":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/"},"author":{"name":"PIA Research","@id":"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/person\/867d81a36eafaf83e91f6528aca0ba29"},"headline":"Linux networking stack from the ground up, part 5","datePublished":"2016-02-03T14:46:08+00:00","dateModified":"2024-02-01T10:16:19+00:00","mainEntityOfPage":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/"},"wordCount":1188,"commentCount":1,"publisher":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/#organization"},"articleSection":["General Privacy News"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/","url":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/","name":"Linux networking stack from the ground up, part 5","isPartOf":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/#website"},"datePublished":"2016-02-03T14:46:08+00:00","dateModified":"2024-02-01T10:16:19+00:00","description":"part 1 | part 2 | part 3 | part 4 | part 5 Overview This blog post picks up right where part 4 left off and begins by examining the last part of the IP","breadcrumb":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.privateinternetaccess.com\/blog\/linux-networking-stack-from-the-ground-up-part-4-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.privateinternetaccess.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Linux networking stack from the ground up, part 5"}]},{"@type":"WebSite","@id":"https:\/\/www.privateinternetaccess.com\/blog\/#website","url":"https:\/\/www.privateinternetaccess.com\/blog\/","name":"PIA","description":"Online privacy news from around the world.","publisher":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.privateinternetaccess.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.privateinternetaccess.com\/blog\/#organization","name":"Private Internet Access","url":"https:\/\/www.privateinternetaccess.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.privateinternetaccess.com\/blog\/wp-content\/uploads\/2018\/07\/pialogowhitekglogo.png","contentUrl":"https:\/\/www.privateinternetaccess.com\/blog\/wp-content\/uploads\/2018\/07\/pialogowhitekglogo.png","width":1200,"height":1200,"caption":"Private Internet Access"},"image":{"@id":"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/privateinternetaccess\/","https:\/\/x.com\/buyvpnservice","https:\/\/www.instagram.com\/piavpn\/","https:\/\/www.youtube.com\/channel\/UClyJZ47Rizb1xnwuKXDI0_w"]},{"@type":"Person","@id":"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/person\/867d81a36eafaf83e91f6528aca0ba29","name":"PIA Research","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.privateinternetaccess.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/717a02042be28ce22f2ae82923cdaa82551205b8768e3cd4a8fce14988ed0ccd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/717a02042be28ce22f2ae82923cdaa82551205b8768e3cd4a8fce14988ed0ccd?s=96&d=mm&r=g","caption":"PIA Research"},"sameAs":["https:\/\/www.privateinternetaccess.com"],"url":"https:\/\/www.privateinternetaccess.com\/blog\/author\/piaresearch\/"}]}},"_links":{"self":[{"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/posts\/1961","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/comments?post=1961"}],"version-history":[{"count":8,"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/posts\/1961\/revisions"}],"predecessor-version":[{"id":18781,"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/posts\/1961\/revisions\/18781"}],"wp:attachment":[{"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/media?parent=1961"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/categories?post=1961"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.privateinternetaccess.com\/blog\/wp-json\/wp\/v2\/tags?post=1961"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}