Telemetry Receiver by UDP+KV-GPB


In the article, only show as follow (because easy to show how to work):

Protocol: UDP; Port: 5234

Encoding: KV-GPB

For more detailed information, you can reference my colleague’s GitHub, which includes more examples, e.g GRPC.

AlexFengCisco / Telemetry_Receiver

GRPC Update Info:

If use GRPC not TCP/UDP, that will have standard function, you will not need to write “DECODE_FN_MAP…”, function will auto generate, direct to use. Simple list steps:

  1. Get GRPC protobuf (e.g “mdt_grpc_dialout”) from cisco / bigmuddy-network-telemetry-proto
  2. Install/use compile tools, suggest install python virtual env, you can reference 在MacOS中部署Python虚拟开发环境
(telemetry-protocol) [root@telemetry telemetry-protocol]# python -V
Python 3.6.8
(telemetry-protocol) [root@telemetry telemetry-protocol]# easy_install pip
(telemetry-protocol) [root@telemetry telemetry-protocol]# pip install grpcio
(telemetry-protocol) [root@telemetry telemetry-protocol]# pip install protobuf
(telemetry-protocol) [root@telemetry telemetry-protocol]# pip install grpcio_tools
(telemetry-protocol) [root@telemetry telemetry-protocol]# python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. cisco_grpc_dialout.proto
(telemetry-protocol) [root@telemetry telemetry-protocol]# ls -l|grep cisco
-rw-r--r--.  1 root    root       2695 Aug 10 16:16   #generated
-rw-r--r--.  1 root    root       3805 Aug 10 16:16   #generated
-rw-r--r--.  1 root    root        485 Aug 10 16:15 cisco_grpc_dialout.proto   #proto file  
Follow all file if you need grpc protocol:
[root@telemetry grpc-dailout]# ls -l
total 36
-rw-r--r--. 1 root root  2695 Aug 10 16:16 #Contains the server Stub class and the client Stub class, as well as the service RPC interface to be implemented.
-rw-r--r--. 1 root root  3805 Aug 10 16:16 #message serialization classes
-rw-r--r--. 1 root root   484 Aug 10 15:36 mdt_grpc_dialout.proto 
-rw-r--r--. 1 root root  3722 Aug 10 16:11 #service python
-rw-r--r--. 1 root root 19220 Aug 10 15:58  #decode gpb-kv messages, generate as below

Install Protobuf of python version

For detailed info, can reference google official doc: Protocol Buffer Basics: Python and Download Protocol Buffers.

[root@telemetry telemetry-protocol]# wget
[root@telemetry telemetry-protocol]# unzip 
[root@telemetry telemetry-protocol]# cd protobuf-3.12.4/
[root@telemetry protobuf-3.12.4]# ./configure 
[root@telemetry protobuf-3.12.4]# make
[root@telemetry protobuf-3.12.4]# make check
PASS: protobuf-test
PASS: protobuf-lazy-descriptor-test
PASS: protobuf-lite-test
PASS: google/protobuf/compiler/
PASS: protobuf-lite-arena-test
PASS: no-warning-test
Testsuite summary for Protocol Buffers 3.12.4
# TOTAL: 6
# PASS:  6
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
[root@telemetry protobuf-3.12.4]# make install
[root@telemetry protobuf-3.12.4]# cd ./python/
[root@telemetry python]# python3 build
[root@telemetry python]# python3 test
[root@telemetry python]# python3 install
[root@telemetry python]# protoc --version
libprotoc 3.12.4

Compile Proto File

In the example, only show KV-GPB, so only require 1 proto file (telemetry.proto) for all sensor-path, if you use GPB for encoding, each sensor-path requires 1 proto file. The XR Proto file download from the official link: cisco / bigmuddy-network-telemetry-proto

[root@telemetry telemetry-protocol]# protoc -I=./ --python_out=./ ./telemetry.proto
[root@telemetry telemetry-protocol]# ls -l
total 6340
drwxr-xr-x. 13 root    root       4096 Aug  9 15:58 protobuf-3.12.4
-rw-r--r--.  1 root    root    6100223 Jul 29 07:58
-rw-r--r--.  1 root    root       7493 Aug  9 15:34
-rw-r--r--.  1 root    root      19220 Aug  9 16:28   <<<
-rw-r--r--.  1 root    root       8187 Aug  9 16:27 telemetry.proto
-rw-r--r--.  1 tcpdump tcpdump  344548 Aug  9 15:15 telemetry-udp.pcap

Telemetry Config On XR

How to get sensor path, please reference my last blog: How To Get Telemetry sensor path for show cmd on IOS XR?

telemetry model-driven
 destination-group nms
  address-family ipv4 port 5432
   encoding self-describing-gpb
   protocol udp
 sensor-group interface
  sensor-path Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface[interface-name='GigabitEthernet0/0/0/1']
 subscription nms
  sensor-group-id interface strict-timer
  sensor-group-id interface sample-interval 20000
  destination-id nms
  source-interface GigabitEthernet0/0/0/2

Telemetry UDP Server by Python

Come From: AlexFengCisco / Telemetry_Receiver

[root@telemetry telemetry-protocol]# more
     |          MSG TYPE             |           ENCODING_TYPE       |
     |         MSG_VERSION           |           FLAGS               |
     |                           MSG_LENGTH                          |
     ~                                                               ~
     ~                      PAYLOAD (MSG_LENGTH bytes)               ~
     MSG TYPE (2 bytes)  = 1 (for MDT)
     ENCODING_TYPE (2 bytes) = 1 (GPB), 2 (JSON)
     MSG_VERSION (2 bytes) = 1
     FLAGS (2 bytes) = 0
     MSG_LENGTH (4 bytes)

import socket, struct
import json
from google.protobuf.descriptor import FieldDescriptor
import time
import pprint
import telemetry_pb2
from google.protobuf.json_format import MessageToJson

    FieldDescriptor.TYPE_DOUBLE: float,
    FieldDescriptor.TYPE_FLOAT: float,
    FieldDescriptor.TYPE_INT32: int,
    FieldDescriptor.TYPE_INT64: int, #long
    FieldDescriptor.TYPE_UINT32: int,
    FieldDescriptor.TYPE_UINT64: int,#long
    FieldDescriptor.TYPE_SINT32: int,
    FieldDescriptor.TYPE_SINT64: int,#long
    FieldDescriptor.TYPE_FIXED32: int,
    FieldDescriptor.TYPE_FIXED64: int,#long
    FieldDescriptor.TYPE_SFIXED32: int,
    FieldDescriptor.TYPE_SFIXED64: int,#long
    FieldDescriptor.TYPE_BOOL: bool,
    FieldDescriptor.TYPE_STRING: str,
    FieldDescriptor.TYPE_BYTES: bytes,#lambda b: bytes_to_string(b),
    FieldDescriptor.TYPE_ENUM: int,

def field_type_to_fn(msg, field):
    if field.type == FieldDescriptor.TYPE_MESSAGE:
        # For embedded messages recursively call this function. If it is
        # a repeated field return a list
        result = lambda msg: proto_to_dict(msg)
    elif field.type in DECODE_FN_MAP:
        result = DECODE_FN_MAP[field.type]
        raise TypeError("Field %s.%s has unrecognised type id %d" % (
                         msg.__class__.__name__,, field.type))
    return result

def proto_to_dict(msg):
    result_dict = {}
    extensions = {}
    for field, value in msg.ListFields():
        conversion_fn = field_type_to_fn(msg, field)

        # Skip extensions
        if not field.is_extension:
            # Repeated fields result in an array, otherwise just call the
            # conversion function to store the value
            if field.label == FieldDescriptor.LABEL_REPEATED:
                result_dict[] = [conversion_fn(v) for v in value]
                result_dict[] = conversion_fn(value)
    return result_dict

# Bind Socket UDP port 5432 as Telemetry recevice server
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(('', 5432))

count = 0

start_time = time.time()

while True:
    count += 1
    buf, addr = sock.recvfrom(65535)
    Telemetry_content = telemetry_pb2.Telemetry()

    print("Message Length {}".format(len(buf)))
    #Handle Telemetry UDP GPB and GPB-kv  from IOX
    if buf[0:1] == b'\x00': ##check the binary daa , no official document
        print("Telemetry GPB  message from IOX")
        print('Node :'+Telemetry_content.node_id_str)
        print('IP Address (source port) :' + str(addr))
        print('Encodig Path :' + Telemetry_content.encoding_path)
        content_json_dict = proto_to_dict(Telemetry_content.data_gpb)  # Old proto_to_dict from google public code
        print(MessageToJson(Telemetry_content)) # new MessageToJson from google public code , will replace proto_to_dict

        if len(str(Telemetry_content.data_gpbkv)) > 2: # Handle GPB kv , in case of unstable A9KV , sometimes sent empty content message
            print('GPB kv format')
            Fields_list = Telemetry_content.data_gpbkv[0].fields[1].fields

            json_dict = proto_to_dict(Telemetry_content.data_gpbkv[0])

            for field in Fields_list:

                if field.string_value:
                    print( + ':' + field.string_value)
                if field.uint32_value:
                    print( + ':' + str(field.uint32_value))


Run the script:

[root@telemetry telemetry-protocol]# python3
Message Length 124 <<<
Telemetry GPB  message from IOX
Node :iosxrv9000-1
IP Address (source port) :('', 45761)
Encodig Path :Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface
  "nodeIdStr": "iosxrv9000-1",
  "subscriptionIdStr": "nms",
  "encodingPath": "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface",
  "collectionId": "3466",
  "msgTimestamp": "1596974694925",
  "collectionEndTime": "1596974694925"
Message Length 2620 <<<
Telemetry GPB  message from IOX
Node :iosxrv9000-1
IP Address (source port) :('', 45761)
Encodig Path :Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface
  "nodeIdStr": "iosxrv9000-1",
  "subscriptionIdStr": "nms",
  "encodingPath": "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface",
  "collectionId": "3466",
  "collectionStartTime": "1596974694910",
  "msgTimestamp": "1596974694910",
  "dataGpbkv": [
      "timestamp": "1596974694924",
      "fields": [
          "name": "keys",
          "fields": [
              "name": "interface-name",
              "stringValue": "GigabitEthernet0/0/0/1"
          "name": "content",
          "fields": [
              "name": "interface-handle",
              "stringValue": "GigabitEthernet0/0/0/1"
GPB kv format
{'timestamp': 1596974694924, 'fields': [{'name': 'keys', 'fields': [{'name': 'interface-name', 'string_value': 'GigabitEthernet0/0/0/1'}]}, {'name': 'content', 'fields': [{'name': 'interface-handle', 'string_value': 'GigabitE}
description:to server-1
^CTraceback (most recent call last):
  File "", line 90, in 
    buf, addr = sock.recvfrom(65535)

Capture packets and checking

[root@telemetry telemetry-protocol]# tcpdump -i ens192 -w telemetry-udp-new.pcap
tcpdump: listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
^C380 packets captured
381 packets received by filter
0 packets dropped by kernel


本文出自 Frank's Blog


本文链接:Telemetry Receiver by UDP+KV-GPB
版权声明:本文为原创文章,仅代表个人观点,版权归 Frank Zhao 所有,转载时请注明本文出处及文章链接
你可以留言,或者trackback 从你的网站


blonde teen swallows videos