隨着雲計算和SDN的應用越來越多,對於SDN控制器的選擇也有許多,例如OpenDaylight、RYU、Floodlight和POX等,我們先選擇RYU與openstack集成作為研究方向。RYU作為Python開發的輕量級SDN控制器,可以非常容易的可以與Openstack集成在一起,現在Openstack集成SDN的方案中使用RYU的項目主要包含以下幾種:
方案1. openstack在Icehouse和Juno版中neutron已經包含RyuNeutronPluginV2作為核心插件實現SDN控制器。
方案2. openstack在Kilo和Liberty版中ryu的插件已經從neutron中移除,而是通過ofagent機制driver來實現RYU控制器的集成
方案3. Dragonflow項目作為一個輕量級集成RYU控制器的SDN項目,實現了L2和分佈式L3等功能。
以上三個基於RYU控制器的sdn解決方案各有特點:
方案1是配置RyuNeutronPluginV2插件,但是如果使用此插件,就無法使用ml2插件,也就無法同時支持多type和mech驅動,方案的可擴展性大大降低,同時新版本openstack也不再使用此解決方案。
方案2是基於ml2插件基礎上,配置ofagent機制驅動來實現RYU的集成,使用ml2插件就可以很好的支持多種type和mech驅動,因此在新版本openstack已經開始推薦使用ofagent機制驅動方式。
方案3是以色列華為團隊獨立開發的基於ryu的sdn控制器項目,它實現了L2、分佈L3、arp和dhcp等多種網絡功能,可以説是基於RYU的SDN方案中比較有特色和有創新的方案。
作為研究RYU控制器與openstack集成的目的,此三種方案都有必要進行一下學習和研究。我將分三種方案的平台分別進行研究,整理出各自方案的特點。
1.OFAgent方案介紹
OFAgent是一個neutron的核心插件,實現了ML2的機制驅動。它的目標是支持一個純Openflow1.3協議的交換機。
OFAgent控制器與openstack集成需要在控制節點(網絡節點)和計算節點部署OFAgent插件agent,來控制br-int集成網橋,為了快速部署OFAgent試驗環境,可以採用devstack來部署ofagent插件環境,以下為控制節點和計算節點部署配置參考
https://wiki.openstack.org/wiki/Neutron/OFAgent
http://docs.openstack.org/developer/devstack/guides/neutron.html
2.OFAgent方案部署情況説明
private網絡 private-subnet 10.0.0.0/24 fd5b:1720:acf7:0:f816:3eff:fe95:7b54
public 網絡public-subnet 172.24.4.0/24 2001:db8::1
port信息:
36515a5c-737c-4045-8efb-9b43e3a67957 10.0.0.2
bd8b208d-2d20-4bc2-a152-8d38b63ae106 10.0.0.1
b6deaa40-290f-4cf4-baec-52c5305368c0 fd5b:1720:acf7::1
55b2e0e0-b9bc-44d8-bae4-45ed3eb4a60f 172.24.4.2
未創建虛機是網絡配置如下:
1)控制節點
port2為(tap36515a5c-73): addr:92:af:57:37:9c:d6
port3為(tapb6deaa40-29): addr:5e:57:c4:1e:d1:7f
port4為(tapbd8b208d-2d): addr:a6:f2:6f:e1:d5:55
port5為(_ofa-tun-vxlan): addr:a2:40:6a:f6:5e:cc
[stack@controller devstack]$ sudo ovs-vsctl show
cb21984d-ee33-4ac7-8e03-0cfb57010b3e
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
Port br-int
Interface br-int
type: internal
Port veth-pub-int
Interface veth-pub-int
error: "could not open network device veth-pub-int (No such device)"
Port "tap36515a5c-73"
Interface "tap36515a5c-73"
Port "tapbd8b208d-2d"
Interface "tapbd8b208d-2d"
Port _ofa-tun-vxlan
Interface _ofa-tun-vxlan
type: vxlan
options: {df_default="true", in_key=flow, local_ip="0", out_key=flow, remote_ip=flow}
Port "tapb6deaa40-29"
Interface "tapb6deaa40-29"
ovs_version: "2.4.0"
[stack@controller devstack]$ sudo ovs-ofctl dump-flows br-int -O openflow13
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=866.458s, table=0, n_packets=14, n_bytes=1652, priority=1,in_port=3 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=866.023s, table=0, n_packets=0, n_bytes=0, priority=1,in_port=2 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=865.610s, table=0, n_packets=0, n_bytes=0, priority=1,in_port=4 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=866.907s, table=0, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=866.905s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=866.460s, table=2, n_packets=0, n_bytes=0, priority=1,tun_id=0x42b actions=write_metadata:0x1/0xfff,goto_table:7
cookie=0x0, duration=866.903s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=866.901s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7
cookie=0x0, duration=866.899s, table=4, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:5
cookie=0x0, duration=866.897s, table=5, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:6
cookie=0x0, duration=866.895s, table=6, n_packets=0, n_bytes=0, priority=1,arp,arp_op=1 actions=CONTROLLER:65509
cookie=0x0, duration=866.893s, table=6, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:7
cookie=0x0, duration=866.890s, table=7, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:8
cookie=0x0, duration=866.452s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:6e:97:bf actions=output:3
cookie=0x0, duration=866.016s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:95:7b:54 actions=output:2
cookie=0x0, duration=865.603s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:d4:aa:a4 actions=output:4
cookie=0x0, duration=866.888s, table=8, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:9
cookie=0x0, duration=866.886s, table=9, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:10
cookie=0x0, duration=866.885s, table=10, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:11
cookie=0x0, duration=866.883s, table=11, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:12
cookie=0x0, duration=866.881s, table=12, n_packets=14, n_bytes=1652, priority=0 actions=goto_table:13
cookie=0x0, duration=865.607s, table=13, n_packets=14, n_bytes=1652, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:3,output:2,output:4
cookie=0x0, duration=866.880s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
2)計算節點
port2為 2(_ofa-tun-vxlan): addr:9e:aa:01:13:76:6f vxlan隧道端口連接網絡節點
[stack@compute1 ~]$ sudo ovs-vsctl show
71ebef35-168e-4726-971a-b5f6184d89d1
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
Port br-int
Interface br-int
type: internal
Port _ofa-tun-vxlan
Interface _ofa-tun-vxlan
type: vxlan
options: {df_default="true", in_key=flow, local_ip="0", out_key=flow, remote_ip=flow}
Port veth-pub-int
Interface veth-pub-int
error: "could not open network device veth-pub-int (No such device)"
[stack@compute1 ~]$ sudo ovs-ofctl dump-flows br-int -O openflow13
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=21806.716s, table=0, n_packets=0, n_bytes=0, priority=1,tun_dst=10.255.255.101,in_port=2 actions=goto_table:2
cookie=0x0, duration=21868.188s, table=0, n_packets=6, n_bytes=468, priority=0 actions=drop
cookie=0x0, duration=21868.186s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=21868.184s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=21868.183s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7
cookie=0x0, duration=21868.181s, table=4, n_packets=0, n_bytes=0, priority=0 actions=goto_table:5
cookie=0x0, duration=21868.180s, table=5, n_packets=0, n_bytes=0, priority=0 actions=goto_table:6
cookie=0x0, duration=21868.178s, table=6, n_packets=0, n_bytes=0, priority=1,arp,arp_op=1 actions=CONTROLLER:65509
cookie=0x0, duration=21868.176s, table=6, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7
cookie=0x0, duration=21789.918s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:95:7b:54 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2
cookie=0x0, duration=21789.117s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:d4:aa:a4 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2
cookie=0x0, duration=21788.094s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:6e:97:bf actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2
cookie=0x0, duration=21868.175s, table=7, n_packets=0, n_bytes=0, priority=0 actions=goto_table:8
cookie=0x0, duration=21868.173s, table=8, n_packets=0, n_bytes=0, priority=0 actions=goto_table:9
cookie=0x0, duration=21868.170s, table=9, n_packets=0, n_bytes=0, priority=0 actions=goto_table:10
cookie=0x0, duration=21868.169s, table=10, n_packets=0, n_bytes=0, priority=0 actions=goto_table:11
cookie=0x0, duration=21806.714s, table=11, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2,goto_table:12
cookie=0x0, duration=21868.167s, table=11, n_packets=0, n_bytes=0, priority=0 actions=goto_table:12
cookie=0x0, duration=21868.166s, table=12, n_packets=0, n_bytes=0, priority=0 actions=goto_table:13
cookie=0x0, duration=21807.120s, table=13, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1
cookie=0x0, duration=21868.164s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
在計算節點compute1上創建testvm1虛機
#nova list
| 88873f79-39f2-4ee9-bf05-4967980fc640 | testvm1 | ACTIVE | - | Running | private=fd5b:1720:acf7:0:f816:3eff:fe1e:e11c, 10.0.0.13 |
#neutron port-list
cfcb5daf-38e4-4d39-a718-4d3ed713b423 | | fa:16:3e:1e:e1:1c | {"subnet_id": "0626d00a-d74c-447f-ac4e-e55ce3b68bce", |
| | | | "ip_address": "10.0.0.13"}
網絡節點br-int流表
[stack@controller devstack]$ sudo ovs-ofctl dump-flows br-int -O openflow13
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=482.694s, table=0, n_packets=13, n_bytes=1534, priority=1,in_port=2 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=482.288s, table=0, n_packets=11, n_bytes=1456, priority=1,in_port=1 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=481.645s, table=0, n_packets=2, n_bytes=140, priority=1,in_port=3 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=285.849s, table=0, n_packets=16, n_bytes=1832, priority=1,tun_dst=10.255.255.100,in_port=5 actions=goto_table:2
cookie=0x0, duration=544.103s, table=0, n_packets=26, n_bytes=2448, priority=0 actions=drop
cookie=0x0, duration=544.101s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=482.696s, table=2, n_packets=16, n_bytes=1832, priority=1,tun_id=0x42b actions=write_metadata:0x1/0xfff,goto_table:7
cookie=0x0, duration=544.096s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=544.094s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7
cookie=0x0, duration=544.092s, table=4, n_packets=26, n_bytes=3130, priority=0 actions=goto_table:5
cookie=0x0, duration=544.091s, table=5, n_packets=26, n_bytes=3130, priority=0 actions=goto_table:6
cookie=0x0, duration=544.089s, table=6, n_packets=2, n_bytes=84, priority=1,arp,arp_op=1 actions=CONTROLLER:65509
cookie=0x0, duration=544.087s, table=6, n_packets=24, n_bytes=3046, priority=0 actions=goto_table:7
cookie=0x0, duration=285.841s, table=7, n_packets=9, n_bytes=1364, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:1e:e1:1c actions=set_field:0x42b->tun_id,set_field:10.255.255.101->tun_dst,output:5
cookie=0x0, duration=544.086s, table=7, n_packets=31, n_bytes=3514, priority=0 actions=goto_table:8
cookie=0x0, duration=482.688s, table=8, n_packets=0, n_bytes=0, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:6e:97:bf actions=output:2
cookie=0x0, duration=482.281s, table=8, n_packets=6, n_bytes=510, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:95:7b:54 actions=output:1
cookie=0x0, duration=481.639s, table=8, n_packets=1, n_bytes=98, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:d4:aa:a4 actions=output:3
cookie=0x0, duration=544.084s, table=8, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:9
cookie=0x0, duration=544.082s, table=9, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:10
cookie=0x0, duration=544.080s, table=10, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:11
cookie=0x0, duration=285.847s, table=11, n_packets=7, n_bytes=826, priority=1,metadata=0x10001/0x10fff actions=set_field:0x42b->tun_id,set_field:10.255.255.101->tun_dst,output:5,goto_table:12
cookie=0x0, duration=543.867s, table=11, n_packets=17, n_bytes=2080, priority=0 actions=goto_table:12
cookie=0x0, duration=543.866s, table=12, n_packets=24, n_bytes=2906, priority=0 actions=goto_table:13
cookie=0x0, duration=481.643s, table=13, n_packets=24, n_bytes=2906, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:2,output:1,output:3
cookie=0x0, duration=543.864s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
計算節點流表:
port1為 1(qvocfcb5daf-38): addr:0e:f9:4b:69:bf:d6 testvm1虛機連接br-int端口
port2為 2(_ofa-tun-vxlan): addr:9e:aa:01:13:76:6f vxlan隧道端口連接網絡節點
[stack@compute1 devstack]$ sudo ovs-ofctl dump-flows br-int -O openflow13
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=324.285s, table=0, n_packets=18, n_bytes=1916, priority=1,in_port=1 actions=write_metadata:0x10001/0x10fff,goto_table:4
cookie=0x0, duration=323.856s, table=0, n_packets=17, n_bytes=2308, priority=1,tun_dst=10.255.255.101,in_port=2 actions=goto_table:2
cookie=0x0, duration=600.046s, table=0, n_packets=3, n_bytes=250, priority=0 actions=drop
cookie=0x0, duration=600.043s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=324.288s, table=2, n_packets=17, n_bytes=2308, priority=1,tun_id=0x42b actions=write_metadata:0x1/0xfff,goto_table:7
cookie=0x0, duration=600.040s, table=2, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x0, duration=600.038s, table=3, n_packets=0, n_bytes=0, priority=0 actions=goto_table:7
cookie=0x0, duration=600.036s, table=4, n_packets=18, n_bytes=1916, priority=0 actions=goto_table:5
cookie=0x0, duration=600.034s, table=5, n_packets=18, n_bytes=1916, priority=0 actions=goto_table:6
cookie=0x0, duration=600.033s, table=6, n_packets=2, n_bytes=84, priority=1,arp,arp_op=1 actions=CONTROLLER:65509
cookie=0x0, duration=600.031s, table=6, n_packets=16, n_bytes=1832, priority=0 actions=goto_table:7
cookie=0x0, duration=323.847s, table=7, n_packets=6, n_bytes=510, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:95:7b:54 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2
cookie=0x0, duration=323.843s, table=7, n_packets=0, n_bytes=0, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:6e:97:bf actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2
cookie=0x0, duration=323.841s, table=7, n_packets=1, n_bytes=98, priority=1,metadata=0x10001/0x10fff,dl_dst=fa:16:3e:d4:aa:a4 actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2
cookie=0x0, duration=600.029s, table=7, n_packets=26, n_bytes=3532, priority=0 actions=goto_table:8
cookie=0x0, duration=324.272s, table=8, n_packets=9, n_bytes=1364, priority=1,metadata=0x1/0xfff,dl_dst=fa:16:3e:1e:e1:1c actions=output:1
cookie=0x0, duration=600.027s, table=8, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:9
cookie=0x0, duration=600.026s, table=9, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:10
cookie=0x0, duration=600.024s, table=10, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:11
cookie=0x0, duration=323.854s, table=11, n_packets=9, n_bytes=1224, priority=1,metadata=0x10001/0x10fff actions=set_field:0x42b->tun_id,set_field:10.255.255.100->tun_dst,output:2,goto_table:12
cookie=0x0, duration=600.023s, table=11, n_packets=8, n_bytes=944, priority=0 actions=goto_table:12
cookie=0x0, duration=600.020s, table=12, n_packets=17, n_bytes=2168, priority=0 actions=goto_table:13
cookie=0x0, duration=324.279s, table=13, n_packets=17, n_bytes=2168, priority=1,metadata=0x1/0xfff,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=output:1
cookie=0x0, duration=600.018s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
3.實驗環境OFAgent運行流表分析:
table
0 in_port檢查表
1 GRE隧道進入表
2 Vxlan隧道進入表
3 vlan和flat物理網絡表(虛機發出)
4 本地流入表
5 arp未知tpa(目標ip地址)轉發表
6 arp應答表
7 隧道發出表
8 本地發出表(發往虛機)
9 vlan和flat物理網絡表(發往虛機)
10 GRE隧道廣播表
11 Vxlan隧道廣播表
12 vlan和flat物理網絡廣播表
13 本地廣播表
4.OFAgent流表設計圖
根據OFAgent項目代碼中flows設計文檔可以總結一下流表設計規則