Openstack, neutron, Can't ping external network
I have followed the Icehouse doc to install a 3 node environment my openstack nodes are builded as virtual machines using libvirt in CentOS
http://docs.openstack.org/icehouse/install-guide/install/yum/content/basics-networking-neutron.html
-controller-node : 10.0.0.11 (management network)
-network-node : 10.0.0.21 (management network), 10.0.1.21 (data network)
-compute-node : 10.0.0.31 (management network), 10.0.1.31 (data network)
-external network : 192.168.125.0/24
-demo network : 172.30.1.0/24
In my network node, external NIC is configured witout IP as per the documentation.(PROMISC="yes")
DEVICE="eth1"
TYPE="Ethernet"
ONBOOT="yes"
PROMISC="yes"
BOOTPROTO="none"
HWADDR="52:54:00:D3:92:E2"
UUID="7f8a9e99-fbd0-4c59-900f-2369c9e8f780"
However, after adding networking service. I cannot ping the external network in a below stage.
1) inside(tenant router gateway on network-node) -> outside(external geteway)
[root@network-node ~]# ping 192.168.125.254
PING 192.168.125.254 (192.168.125.254) 56(84) bytes of data.
From 192.168.125.54 icmp_seq=1 Destination Host Unreachable
From 192.168.125.54 icmp_seq=2 Destination Host Unreachable
From 192.168.125.54 icmp_seq=3 Destination Host Unreachable
2) outside -> inside
[root@desktop ~]# ping 192.168.125.150
PING 192.168.125.150 (192.168.125.150) 56(84) bytes of data.
--> receive no response
This is the info of my environment:
1) controller node
###The status of the port for external is "DOWN"###
[root@controller-node]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:91:4e:06 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
inet6 fe80::5054:ff:fe91:4e06/64 scope link
valid_lft forever preferred_lft forever
[root@controller-node]#
[root@controller-node]# neutron net-list
+--------------------------------------+----------+-------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------+-------------------------------------------------------+
| 9cebb2a6-fd73-4ef7-81d2-188652f57ecd | demo-net | c66648c9-c34b-4806-af39-3c982378a411 172.30.1.0/24 |
| e5f7b93c-475c-4c9d-95e4-8d1cf7728013 | ext-net | a1e1fcc6-d596-4959-8923-9b46d64445af 192.168.125.0/24 |
+--------------------------------------+----------+-------------------------------------------------------+
[root@controller-node]# neutron subnet-list
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
| a1e1fcc6-d596-4959-8923-9b46d64445af | ext-subnet | 192.168.125.0/24 | {"start": "192.168.125.150", "end": "192.168.125.159"} |
| c66648c9-c34b-4806-af39-3c982378a411 | demo-subnet | 172.30.1.0/24 | {"start": "172.30.1.2", "end": "172.30.1.254"} |
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron port-list
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| 9810105a-edf5-41bc-a140-81ccf71f6bc4 | | fa:16:3e:34:fd:cb | {"subnet_id": "a1e1fcc6-d596-4959-8923-9b46d64445af", "ip_address": "192.168.125.150"} |
| 98c762ea-d7f7-4c1d-9b74-73efc9990236 | | fa:16:3e:cb:0c:11 | {"subnet_id": "c66648c9-c34b-4806-af39-3c982378a411", "ip_address": "172.30.1.1"} |
| f5eec840-e629-448b-ba9a-fbcd60501247 | | fa:16:3e:ae:a6:fa | {"subnet_id": "c66648c9-c34b-4806-af39-3c982378a411", "ip_address": "172.30.1.2"} |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron port-show 9810105a-edf5-41bc-a140-81ccf71f6bc4
+-----------------------+----------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+----------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | os-network |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| device_id | 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 |
| device_owner | network:router_gateway |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "a1e1fcc6-d596-4959-8923-9b46d64445af", "ip_address": "192.168.125.150"} |
| id | 9810105a-edf5-41bc-a140-81ccf71f6bc4 |
| mac_address | fa:16:3e:34:fd:cb |
| name | |
| network_id | e5f7b93c-475c-4c9d-95e4-8d1cf7728013 |
| security_groups | |
| status | DOWN |
| tenant_id | |
+-----------------------+----------------------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron router-show demo-router
+-----------------------+-----------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------+
| admin_state_up | True |
| external_gateway_info | {"network_id": "e5f7b93c-475c-4c9d-95e4-8d1cf7728013", "enable_snat": true} |
| id | 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 |
| name | demo-router |
| routes | |
| status | ACTIVE |
| tenant_id | c94f1dc5870a4d06a8b6ba947e1ac554 |
+-----------------------+-----------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron router-list
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
| 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 | demo-router | {"network_id": "e5f7b93c-475c-4c9d-95e4-8d1cf7728013", "enable_snat": true} |
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]#
2) network node
[root@network-node ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:f6:31:07 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.21/24 brd 10.0.0.255 scope global eth0
inet6 fe80::5054:ff:fef6:3107/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:d3:92:e2 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5054:ff:fed3:92e2/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:48:c8:65 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.21/24 brd 10.0.1.255 scope global eth2
inet6 fe80::5054:ff:fe48:c865/64 scope link
valid_lft forever preferred_lft forever
5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether ea:8e:aa:ad:57:60 brd ff:ff:ff:ff:ff:ff
6: br-ex: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether a2:f1:0b:6b:34:4f brd ff:ff:ff:ff:ff:ff
inet6 fe80::a0f1:bff:fe6b:344f/64 scope link
valid_lft forever preferred_lft forever
9: br-int: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 32:a4:53:15:fc:4f brd ff:ff:ff:ff:ff:ff
inet6 fe80::30a4:53ff:fe15:fc4f/64 scope link
valid_lft forever preferred_lft forever
12: gre0: <NOARP> mtu 1476 qdisc noop state DOWN
link/gre 0.0.0.0 brd 10.0.0.31
13: gretap0: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
22: tun0@NONE: <POINTOPOINT,NOARP> mtu 1476 qdisc noqueue state DOWN
link/gre 0.0.0.0 peer 10.0.0.31
inet 10.0.1.21 peer 10.0.1.31/32 scope global tun0
24: br-tun: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether de:a8:a4:b1:b1:46 brd ff:ff:ff:ff:ff:ff
inet6 fe80::dc0e:8cff:fe67:d352/64 scope link
valid_lft forever preferred_lft forever
[root@network-node ~]#
[root@network-node ~]# ovs-vsctl show
23804a8f-7c89-4422-9b9f-67bf26a34c51
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "qr-98c762ea-d7"
tag: 1
Interface "qr-98c762ea-d7"
type: internal
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tapf5eec840-e6"
tag: 1
Interface "tapf5eec840-e6"
type: internal
Bridge br-ex
Port "eth1"
Interface "eth1"
Port br-ex
Interface br-ex
type: internal
Port "qg-9810105a-ed"
Interface "qg-9810105a-ed"
type: internal
Bridge br-tun
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "gre-0a00011f"
Interface "gre-0a00011f"
type: gre
options: {in_key=flow, local_ip="10.0.1.21", out_key=flow, remote_ip="10.0.1.31"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "1.11.0"
[root@network-node ~]#
[root@network-node ~]# ip netns list
qdhcp-9cebb2a6-fd73-4ef7-81d2-188652f57ecd
qrouter-8ae4b1fa-fb60-4690-bbe2-febbfbcf7555
[root@network-node ~]#
[root@network-node ~]# ip netns exec qrouter-8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.30.1.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-98c762ea-d7
192.168.125.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-9810105a-ed
0.0.0.0 192.168.125.254 0.0.0.0 UG 0 0 0 qg-9810105a-ed
[root@network-node ~]#
Any hints would be much appreciated!
Cheers,
hbseo
Answer
I know it is late... but let me try to answer. Not being able to ping the external network might be related to the virtualization and promiscuous mode but I think that step should not work. I will try to explain why. For a very good explanation on how the networking is setup you can check out this presentation: https://www.hastexo.com/system/files/neutron_packet_flows-notes-handout.pdf
You have multiple parallel network spaces involved here.
First you have the global or main routing table of the host. In the case of the network node you have 3 ethernet interface, two with and IP address and one without.
Second you have your external network. This is not configured on your network host. If you would run
netstat -rn
you would not see any route towards your external network. This means that when you send you ICMP packets to the external gateway they would go the default gateway set on the network node and not be answered. This network lives in the neutron router's network namespace.
Third you have the tenant network which also does not live in your main network space but in the same neutron router namespace.
To exemplify I will show you how it looks in my setup. I have an all in one node that is running inside a Virtualbox machine and it is running Ubuntu Trusty. My interfaces are eth0 for admin (also my default gateway connected to a NAT network in Virtualbox), eth1 for the tunnels, eth2 is my external net (also attached to a NAT network with the range 172.16.100.0/24).
Here is how the routing table of the Ubuntu VM looks like:
root@columbo:~# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.1.1 0.0.0.0 UG 0 0 0 eth0
10.0.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.56.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
So no sign of my external net (172.16.100.1). If I try to ping it I get nothing:
ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
^C
--- 172.16.100.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2004ms
Now you can see what network namespaces you have using the command ip netns list
root@columbo:~# ip netns list
qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b
qdhcp-120a6fde-7e2d-4856-90ee-5609a5f3035f
qdhcp-b7ab2080-a71a-44f6-9f66-fde526bb73d3
Inside a namespace you can run most of your normal commands - see bellow
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 172.16.100.1 0.0.0.0 UG 0 0 0 qg-c69702a9-ae
10.255.1.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-e706aba1-46
172.16.100.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-c69702a9-ae
and
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b arp -an
? (10.255.1.12) at fa:16:3e:40:50:36 [ether] on qr-e706aba1-46
? (172.16.100.1) at 52:54:00:12:35:00 [ether] on qg-c69702a9-ae
? (10.255.1.14) at fa:16:3e:75:5f:b4 [ether] on qr-e706aba1-46
and now ping will work (inside the namespace)
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
64 bytes from 172.16.100.1: icmp_seq=1 ttl=255 time=2.55 ms
64 bytes from 172.16.100.1: icmp_seq=2 ttl=255 time=0.555 ms
^C
--- 172.16.100.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.555/1.554/2.553/0.999 ms
I can also ping the router which has the ip 172.16.100.50:
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 172.16.100.50
PING 172.16.100.50 (172.16.100.50) 56(84) bytes of data.
64 bytes from 172.16.100.50: icmp_seq=1 ttl=64 time=0.065 ms
64 bytes from 172.16.100.50: icmp_seq=2 ttl=64 time=0.065 ms
^C
--- 172.16.100.50 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.065/0.065/0.065/0.000 ms
Same for my tenant network
root@columbo:~# neutron net-list
+--------------------------------------+---------------+------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+---------------+------------------------------------------------------+
| 120a6fde-7e2d-4856-90ee-5609a5f3035f | SecondVlan | 5432f1c9-0bb6-4619-b897-65d301071f72 5.5.5.0/25 |
| f2597437-a005-44ad-9ce2-168fbc331e56 | outside_world | 3fe35e71-53d7-4432-8c82-a06856b79316 172.16.100.0/24 |
| b7ab2080-a71a-44f6-9f66-fde526bb73d3 | SERVER_VLAN_1 | 87d769f1-5cf3-48cf-8741-44a01479ff3e 10.255.1.0/24 |
+--------------------------------------+---------------+------------------------------------------------------+
the ping of the tenant router in the SERVER vlan
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 10.255.1.1
PING 10.255.1.1 (10.255.1.1) 56(84) bytes of data.
64 bytes from 10.255.1.1: icmp_seq=1 ttl=64 time=0.050 ms
64 bytes from 10.255.1.1: icmp_seq=2 ttl=64 time=0.064 ms
^C
--- 10.255.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.050/0.057/0.064/0.007 ms
I hope it is useful for somebody and not totally flawed.