大网数据广告活动
服务器运维问答

DNAT without default route

· 52CE 我爱测

I have a TCP service in a datacenter that is doing filtering and rate limiting based on source IP address. I'd like to move it to another datacenter. I'd like to provide the same service on an IP address from the new datacenter a…

问题原文

原帖链接 · Tometzky · CC BY-SA 4.0

I have a TCP service in a datacenter that is doing filtering and rate limiting based on source IP address. I'd like to move it to another datacenter.

I'd like to provide the same service on an IP address from the new datacenter and forward all traffic on a single port to the old one, so both new and old will work at the same time. I can't just change the hostname, as some clients are using IP address for connecting (sigh) and some are using outgoing connections IP filtering based on IP address (sigh) and it will take weeks to change them.

I know that I can SNAT the connections, but if I do this all the connections will be sourced from the same IP, which conflicts with filtering and rate limiting based on source IP address.

I can DNAT the connections and route them through a VPN tunnel, but this means that the return packets will try to go with the service's default route and service source IP address and will be ignored by the clients.

Is there a way with Linux to somehow mark the TCP packets that were DNAT'ed so the return packets can be routed back through the VPN tunnel instead of the service's default route?

已采纳解答

原帖链接 · Anton Danilov · CC BY-SA 4.0

There are various ways to implement what you want. Draw your network topology.

Simplest way

It requires only single DNAT rule on the S2 (server in the new DC) and additinal routing configuration on the S1 (server in the old DC). But it also requires what your app accept the requests on the VPN tunnel address too.

The S2 server iptables configuration:

iptables -t nat -A PREROUTING \
         -i eth0 --dst <S2.IP> \
         -p tcp --dport <APP.PORT> \
    -j DNAT --to-address <S1.TUN.IP>:<APP.PORT>

Also, you should enable the forwarding on the S2 server (use the sysctl -w net.ipv4.ip_forward=1 command to enable it).

Verification: use the ip route get <S1.TUN.IP> from 8.8.8.8 iif <S2.IFACE> and ip route get 8.8.8.8 from <S1.TUN.IP> iif <S2.TUN.IFACE> command. It should return the valid routes.

The S1 server routing configuration:

ip route add 0/0 dev <TUN.IFACE> table 1
ip rule add from <S1.TUN.IP> lookup 1 pref 1000

LINUX replies on the request from the same ip address, on what request has been received.

Verification: use the ip route get <S1.TUN.IP> from 8.8.8.8 iif <S1.TUN.IFACE> and ip route get 8.8.8.8 from <S1.TUN.IP> commands. It also should return the valid routes. Maybe you will see something like invalid cross-device link. In this case you should tune the rp_filter on the vpn tunnel interface.

Detailed explanation:

  1. Client sends the request in form of <C.IP>:<SOME.PORT> -> <S2.IP>:<APP.PORT>.
  2. S2 server receives this request, rewrites the destination to <S1.TUN.IP>. It happens before routing, so after this step the packet will form of <C.IP>:<SOME.PORT> -> <S1.TUN.IP>:<APP.PORT>.
  3. S2 forwards the rewritten request through VPN tunnel due the routing table.
  4. S1 receives the request through VPN tunnel to <S1.TUN.IP> address.
  5. Your app on S1 serves the request and replies to client with source address <S1.TUN.IP>. The reply is <S1.TUN.IP>:<APP.PORT> -> <C1.IP>:<SOME.PORT>.
  6. By routing rule all packets with source address <S1.TUN.IP> routes by the routing table 1. So, the replied packets from your app will be sent through VPN tunnel to S2 server.
  7. S2 receives the replies, make reverse translation of source address, rewriting it from <S1.TUN.IP> into <S2.IP>. After this reply becomes into <S2.IP>:<APP.PORT> -> <C.IP>:<SOME.PORT>.
  8. The rewritten replies are being forwarded back to client to <C.IP> destination address.
  9. The client receives the reply as expected.

To troubleshoot you can use the tcpdump.

There is other way, that is more complicated. I'll describe it if you need.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2019-05-23)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。