TCP option
, and the client can obtain the real client IP address by calling the API of the TOA kernel module.TCP option
.modprobe toa
to load TOA. If the server is run on Linux, Linux TOA should be used instead.cd
command to access the decompressed folder and run the following module loading command:insmod toa.ko
dmesg -T | grep TOA
toa.ko
file in the startup script (the toa.ko
file needs to be reloaded if the server is restarted).rmmod toa
dmesg -T
toa.ko
file. This general version supports most Linux distributions (e.g., CentOS 7, CentOS 8, Ubuntu 16.04, and Ubuntu 18.04).wget "https://clb-toa-1255852779.file.myqcloud.com/tgw_toa_linux_ver.tar.gz"
wget "https://clb-toa-1255852779.file.myqcloud.com/tgw_toa_tlinux_ver.tar.gz"
yum install gcc yum install make //Install the kernel development package. The version of the package header file and library must be consistent with the kernel version. yum install kernel-devel-`uname -r`yum install devtoolset-8
apt-get install gcc apt-get install make //Install the kernel development package. The version of the package header file and library must be consistent with the kernel version. apt-get install linux-headers-`uname -r`apt-get install devtoolset-8
zypper install gcczypper install make//Install the kernel development package. The version of the package header file and library must be consistent with the kernel version.zypper install kernel-default-develzypper install devtoolset-8
PATH=/opt/rh/devtoolset-8/root/bin:$PATH
. Before compiling, make sure that the Kernel version matches the GCC version. You can run dmesg | grep 'Linux version'
to check the Kernel version. toa.ko
file. If warning
and error
are not prompted during the compilation process, the compilation is successful. Take the source package for Linux OS as an example:tar zxvf tgw_toa_linux_ver.tar.gzcd tgw_toa_linux_ver//Enter the decompressed directory tgw_toamake
struct sockaddr v4addr;len = sizeof(struct sockaddr);//`get_peer_name` is the standard Linux network programming API.if (get_peer_name(client_fd, &v4addr, &len) == 0) {inet_ntop(AF_INET, &(((struct sockaddr_in *)&v4addr)->sin_addr), from, sizeof(from));printf("real client v4 [%s]:%d\\n", from, ntohs(((struct sockaddr_in *)&v4addr)->sin_port));}
toa.ko
kernel module is inserted into the real server and TOA will pass the IP address to the real server.struct toa_nat64_peer {struct in6_addr saddr;uint16_t sport;};....struct toa_nat64_peer client_addr;....
enum {TOA_BASE_CTL = 4096,TOA_SO_SET_MAX = TOA_BASE_CTL,TOA_SO_GET_LOOKUP = TOA_BASE_CTL,TOA_SO_GET_MAX = TOA_SO_GET_LOOKUP,};getsockopt(client_fd, IPPROTO_IP, TOA_SO_GET_LOOKUP, &client_addr, &len);
real_ipv6_saddr = client_addr.saddr;real_ipv6_sport = client_addr.sport;
//Define TOA variables. Set `TOA_BASE_CTL` to 4096.enum {TOA_BASE_CTL = 4096,TOA_SO_SET_MAX = TOA_BASE_CTL,TOA_SO_GET_LOOKUP = TOA_BASE_CTL,TOA_SO_GET_MAX = TOA_SO_GET_LOOKUP,};//Define a data structure to store the IP address.struct toa_nat64_peer {struct in6_addr saddr;uint16_t sport;};//Declare the variable that is used to store the address.struct toa_nat64_peer client_addr;.……//Get the file descriptor of the client, where `listenfd` is the listening file descriptor of the server.client_fd = accept(listenfd, (struct sockaddr*)&caddr, &length);//Make calls to get the real client IP address of the user in the NAT64 scenario.char from[40];int len = sizeof(struct toa_nat64_peer);if (getsockopt(client_fd, IPPROTO_IP, TOA_SO_GET_LOOKUP, &client_addr, &len) == 0) {inet_ntop(AF_INET6, &client_addr.saddr, from, sizeof(from));//Obtain the source IP and source portprintf("real client [%s]:%d\\n", from, ntohs(client_addr.sport));}
toa.ko
kernel module is inserted into the real server and TOA will pass the IP address to the real server.//Define TOA variables. Set `TOA_BASE_CTL` to 4096.enum {TOA_BASE_CTL = 4096,TOA_SO_SET_MAX = TOA_BASE_CTL,TOA_SO_GET_LOOKUP = TOA_BASE_CTL,TOA_SO_GET_MAX = TOA_SO_GET_LOOKUP,};//Define a data structure to store the IP address.struct toa_nat64_peer {struct in6_addr saddr;uint16_t sport;};//Declare the variable that is used to store the address.struct toa_nat64_peer client_addr_nat64;.......//Get the file descriptor of the client, where `listenfd` is the listening file descriptor of the server.//Make calls to get the real client IP in the NAT64 scenario.char from[40];int len = sizeof(struct toa_nat64_peer);int ret;ret = getsockopt(client_fd, IPPROTO_IP, TOA_SO_GET_LOOKUP, &client_addr_nat64, &len);if (ret == 0) {inet_ntop(AF_INET6, &(client_addr_nat64.saddr), from, sizeof(from));//Obtain the source IP and source port.printf("real client v6 [%s]:%d\\n", from, ntohs(client_addr_nat64.sport));} else if (ret != 0) {struct sockaddr v4addr;len = sizeof(struct sockaddr);//Obtain the source IP and source port://In the hybrid cloud deployment scenario, the source IP address is the IP address after SNAT;//In the non-hybrid cloud deployment scenario, the source IP address is the client IP address without SNAT and NAT64.//The semantics of this function is to get the real client address and port.if (get_peer_name(client_fd, &v4addr, &len) == 0) {inet_ntop(AF_INET, &(((struct sockaddr_in *)&v4addr)->sin_addr), from, sizeof(from));printf("real client v4 [%s]:%d\\n", from, ntohs(((struct sockaddr_in *)&v4addr)->sin_port));}}
toa.ko
kernel module, you can monitor the TOA working status on the host of the container in either of the following ways.cat /proc/net/toa_table
cat /proc/net/toa_stats
Metric | Description |
syn_recv_sock_toa | Receives connections with TOA information. |
syn_recv_sock_no_toa | Receives connections without TOA information. |
getname_toa_ok | This count increases when you call getsockopt and obtain the source IP address successfully or when you call accept to receive client requests. |
getname_toa_mismatch | This count increases when you call getsockopt and obtain a source IP address that does not match the required type. For example, if a client connection contains a source IPv4 address whereas you obtain an IPv6 address, the count will increase. |
getname_toa_empty | This count increases when the getsockopt function is called in a client file descriptor that does not contain TOA. |
ip6_address_alloc | Allocates space to store the information when TOA obtains the source IP address and source port saved in the TCP data packet. |
ip6_address_free | When the connection is released, TOA will release the memory previously used to save the source IP and source port. If all connections are closed, the total count of ip6_address_alloc for each CPU should be equal to the count of this metric. |
tlinux
in the command output, you are using TLinux OS, while linux
indicates that you are using Linux OS.uname -a
tlinux
or tl2
is returned, you are using TLinux OS.rpm -qa | grep kernel
lsmod | grep toa
unknown-200
is displayed in the tcp option
output, the real client IP address after SNAT is inserted into the field tcp option
.unknown-253
is displayed, the real client IPv6 address is inserted in the NAT64 scenario.
toa.ko
to a DEBUG version, and further locate the problem through kernel logs. In the downloaded TOA source directory, add the DEBUG compilation option to the make file.make cleanmake
toa.ko
file and install the latest one.rmmod toainsmod ./toa.ko
dmesg -Tw
Was this page helpful?