大网数据广告活动
服务器运维问答

Routing from one server to another with different interfaces

· 52CE 我爱测

I have this topology Server A: eth0 IP: 10.10.20.11/24 and 10.10.20.13/24 eth3: 88.x.x.x (vlan with id 10) primary gateway: GW2 ( 10.10.20.3 ) Server B: eth0 IP: 10.10.20.23/24 default gateway: server A ( 10.10.20.11 ) From outsi…

问题原文

原帖链接 · Kordi · CC BY-SA 4.0

I have this topology

Server A:

  • eth0 IP: 10.10.20.11/24 and 10.10.20.13/24
  • eth3: 88.x.x.x (vlan with id 10)
  • primary gateway: GW2 (10.10.20.3)

Server B:

  • eth0 IP: 10.10.20.23/24
  • default gateway: server A (10.10.20.11)

From outside I can ping the 88.x.x.x IP. Server A can ping Server B. On Server B I disabled the firewall.

Now I am trying to route traffic from 88.x.x.x Server A to Server B with the following rules:

<?xml version="1.0" encoding="utf-8"?>
<direct>
    <passthrough ipv="ipv4">-A PREROUTING  -t nat -p tcp -d 88.x.x.x   --dport X -j DNAT  --to-destination 10.10.10.23:Y</passthrough>
</direct>

Sysctl variables on server A: During troubleshooting I've got some results:

  • tcpdump port X on Server A shows packets
  • no packets were shown for tcpdump dst 10.10.10.23 on Server A.
  • tcpdump port Y on Server B doesn't show anything.
  • The following works on server a: telnet 10.10.10.23 Y

After disabling rp_filter I've got half-working topology - At least I've seen the redirected and DNATted packets in the tcpdump.

After this I've got following picture:

  • Server A: tcpdump -i any port X -n
IP 85.x.x.x > 88.x.x.x.X
  • Server A: tcpdump -i any port Y -n
IP 85.x.x.x > 10.10.10.23.Y:
  • Server B: tcpdump -i any port Y -n
IP 85.x.x.x > 10.10.10.23.Y

I had extended the tcpdump filter to port Y and icmp and I've started see the icmp admin prohibited messages in the replies from 10.10.10.23 host. Then I've tuned the firewall rules on server B - icmp messages stopped.

Currently I have this routing configurations: - Server A

ip -4 ru ls:
0: from all lookup local
32764: from 88.XX/27 lookup 2014
32765: from 10.10.20.11 lookup 1000
32766: from all lookup main
32767: from all lookup default

ip -4 r ls table all:
default via GW1 dev eth3 table 2014
88.x.x.x/27 dev eth3 table 2014
default via 10.10.20.3 dev eth0 table 1000
default via 10.10.20.3 dev eth0
10.10.20.0/24 dev eth0 proto kernel scope link src 10.10.20.11
88.x.x.x/27 dev eth3 proto kernel scope link src 88.x.x.x
broadcast 10.10.20.0 dev eth0 table local proto kernel scope link src 10.10.20.11
local 10.10.20.11 dev eth0 table local proto kernel scope host src 10.10.20.11
local 10.10.20.13 dev eth0 table local proto kernel scope host src 10.10.20.11
broadcast 10.10.20.255 dev eth0 table local proto kernel scope link src 10.10.20.11
broadcast 88.x.x.x dev eth3 table local proto kernel scope link src 88.x.x.x
local 88.x.x.x dev eth3 table local proto kernel scope host src 88.x.x.x
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
  • Server B
ip -4 r ls:
default via 10.10.20.11 dev eth0
10.10.20.0/24 dev eth0 proto kernel scope link src 10.10.20.23

I need to get Access to Server B through real IP (88.x.x.x) from outside. What I am doing wrong or missing?

已采纳解答

原帖链接 · Anton Danilov · CC BY-SA 4.0

Step to troubleshoot:

  1. Check the actual routing. On server A run ip route get 10.10.10.23 from 8.8.8.8 iif eth1. You should see the valid route in the output of this command. If you see something RTNETLINK answers: No route to host, you have disabled forwarding. See the next step. Output like RTNETLINK answers: Invalid cross-device means rp_filter filters similar packets. You also can disable it completely or set into loose mode.

  2. On server A check the forwarding (ip netconf show dev <iface>). It should be enabled (forwarding on in the output). You can turn on it withsysctl` command.

  3. In the SNAT rule you should use --to-source 10.10.10.Z, where 10.10.10.Z is address of eth0 interface of server A. Otherwise there are side effects at translation of reply packets from the server B. If the default gateway of serverB is serverA, then SNAT rule isn't necessary.

  4. Check the full ruleset on server A. Use iptables-save -c command to list all rules with counters. You should see non-zero counters in the rules of FORWARD chain.

  5. Execute the similar steps for serverB: check the routing, check the firewall rule set, check the tcpdump output.

  6. Run tcpdump on both hosts and check the traffic. The tcpdump captures incoming packets before firewall, but outgoing packets - after. Better include icmp in the tcpdump filters, otherwise you won't see the icmp rejects like port unreachable and similar.

  7. When all step above is done, you should add these firewall rules into configuration of server A to make the replied packets from server B pass through GW1:

iptables -t mangle -A FORWARD\
         -p tcp --dst 10.10.20.23 \
         -m conntrack --ctstate DNAT --ctorigdst 88.x.x.x --ctdir ORIGINAL \
    -j CONNMARK --set-mark 2014

iptables -t mangle -A PREROUTING \
         -i eth0 -p tcp \
         -m conntrack --ctstate DNAT --ctdir REPLY \
    -j CONNMARK --restore-mark

And add an additional routing rule into PBR:

ip rule add fwmark 2014 lookup 2014

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2019-05-18)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。