SONA is a project implements OpenStack Neutron ML2 mechanism driver and L3 service plugin with ONOS(onosproject.org). See https://wiki.onosproject.org/display/ONOS/SONA%3A+DC+Network+Virtualization for the details. Here it describes how to set up a gateway node for SONA.
SONA gateway node is composed of a couple of bridge controlled by ONOS-SONA and ONOS-vRouter respectively. SONA ONOS takes care of creating the bridges and patch link between the bridges, and flow rules on the br-int bridge. vRouter ONOS, on the other hand, takes care of flow rules on br-router and communications with external routers.
Prepare the network configuration file for SONA with the information about compute and gateway nodes. There is an example named with sona.json in this repository. Now activate SONA applications and push the network configuration file to running ONOS.
Note that routerController field will be used to bring up ONOS-vRouter later. This address does not span gateway nodes, that is, you can use the same address in multiple gateway nodes.
# activate applications
$ curl --user onos:rocks -X POST http://onos_ip:8181/onos/v1/applications/org.onosproject.drivers/active
$ curl --user onos:rocks -X POST http://onos_ip:8181/onos/v1/applications/org.onosproject.openflow-base/active
$ curl --user onos:rocks -X POST http://onos_ip:8181/onos/v1/applications/org.onosproject.openstackswitching/active
$ curl --user onos:rocks -X POST http://onos_ip:8181/onos/v1/applications/org.onosproject.openstackrouting/active
# push network config
$ curl --user onos:rocks -X POST -H "Content-Type: application/json" http://onos_ip:8181/onos/v1/network/configuration/ -d @sona.json
Check the nodes states are COMPLETE. Use openstack-node-check command for more detailed states of the node. Pushing network configuration triggers reinitialization of the nodes. It's no harm to reinitialize COMPLETE state node. If you want to reinitialize a particular compute node, use openstack-node-init command with hostname.
onos> openstack-nodes
hostname=compute-01, type=COMPUTE, managementIp=10.203.25.244, dataIp=10.134.34.222, intBridge=of:00000000000000a1, routerBridge=Optional.empty init=COMPLETE
hostname=compute-02, type=COMPUTE, managementIp=10.203.229.42, dataIp=10.134.34.223, intBridge=of:00000000000000a2, routerBridge=Optional.empty init=COMPLETE
hostname=gateway-01, type=GATEWAY, managementIp=10.203.198.125, dataIp=10.134.33.208, intBridge=of:00000000000000a3, routerBridge=Optional[of:00000000000000b1] init=COMPLETE
hostname=gateway-02, type=GATEWAY, managementIp=10.203.198.131, dataIp=10.134.33.209, intBridge=of:00000000000000a4, routerBridge=Optional[of:00000000000000b2] init=COMPLETE
Total 4 nodes
vRouter ONOS
Prepare network configuration file for vRouter with external connection information. One example is vrouter.json in this repository. Modify the file as you want.
For more details about vRouter, check out https://wiki.onosproject.org/display/ONOS/vRouter.
When vrouter.json file is ready, now run vrouter.sh script with the routerController IP address to bring up ONOS-vRouter container. The same command can be used to re-run the container.
$ vrouter.sh
If you want specific IP address, add it to the command. Note that this IP address is not managed by Docker so please be careful not to comflict with the other docker containers.
$ vrouter.sh 172.17.0.2
Quagga for vRouter
Modify volumes/gateway/zebra.conf and volumes/gateway/bgpd.conf as you want. Note that fpm connection ip in zebra.conf should be the same with routerController.
Run Quagga container with the IP address, which equals to router-id in bgpd.conf and any MAC address. This MAC address will be used in vrouter.json later.
$ ./quagga.sh --name=gateway-01 --ip=172.18.0.254/24 --mac=fe:00:00:00:00:01
If you check the result of ovs-vsctl show, there should be a new port named quagga on br-router bridge.
External Router
If there's no external router in your setup, add another quagga container in ecah gateway node, which acts as an external router.
Modify volumes/router/zebra.conf and volumes/router/bgpd.conf as you want, and use the same command above but with additional argument --external-router to bring up the router container.
$ ./quagga.sh --name=router-01 --ip=172.18.0.1/24 --mac=fa:00:00:00:00:01 --external-router
Check ports from vRouter-ONOS.
$ ssh -p 8101 karaf@172.17.0.2
# password is karaf
onos> ports
id=of:00000000000000b1, available=true, role=MASTER, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.3.0, serial=None, driver=softrouter, channelId=172.17.0.1:58292, managementAddress=172.17.0.1, name=of:00000000000000b1, protocol=OF_13
port=local, state=disabled, type=copper, speed=0 , portName=br-router, portMac=e6:a0:79:f9:d1:4a
port=1, state=enabled, type=copper, speed=0 , portName=patch-rout, portMac=fe:da:85:15:b1:bf
port=2, state=enabled, type=copper, speed=10000 , portName=veth1, portMac=a2:fe:d4:6a:e9:c1
port=24, state=enabled, type=copper, speed=10000 , portName=quagga, portMac=06:96:1b:36:32:77
port=25, state=enabled, type=copper, speed=10000 , portName=quagga-router, portMac=ea:1e:71:d1:fd:81
If any port number does not match to the ones in vrouter.json, modify the config file with the correct port numbers.
- port number of
quagga->controlPlaneConnectPointof router config - port number of
quagga-routerorveth1(uplink insona.jsonif you have external router) ->portsof interface with172.18.0.254/24IP address. - port number of
patch-rout->portsof interface with172.27.0.254/24, explain this interface later. Same port number goes to thehostslocation with IP172.27.0.1.
Once you modifyvrouter.json, re-run the ONOS-vRouter.
$ vrouter.sh 172.17.0.2
If everything's right, check fpm-connections, hosts and routes. 172.18.0.1 is the external default gateway in this example. The host with IP address 172.27.0.1 is for the internal network which will explain later.
onos> hosts
id=FA:00:00:00:00:01/None, mac=FA:00:00:00:00:01, location=of:00000000000000b1/25, vlan=None, ip(s)=[172.18.0.1]
id=FE:00:00:00:00:01/None, mac=FE:00:00:00:00:01, location=of:00000000000000b1/24, vlan=None, ip(s)=[172.18.0.254]
id=FE:00:00:00:00:02/None, mac=FE:00:00:00:00:02, location=of:00000000000000b1/1, vlan=None, ip(s)=[172.27.0.1], name=FE:00:00:00:00:02/None
onos> fpm-connections
172.17.0.2:52332 connected since 6m ago
onos> next-hops
ip=172.18.0.1, mac=FA:00:00:00:00:01, numRoutes=1
onos> routes
Table: ipv4
Network Next Hop
0.0.0.0/0 172.18.0.1
Total: 1
Table: ipv6
Network Next Hop
Total: 0
Register internal network
Now let's add routes for the internal public network, 172.27.0.0/24 in this example. This network might be the floating IP range in Neutron data model.
First, define fake host and interface for the internal network gateway to the network config file and re-run ONOS-vRouter. The port number should equal to the one of patch-rout port. (Example vrouter.json already has the configuration and you don't need to do it again if it's already set correctly)
# vrouter.json
"hosts" : {
"fe:00:00:00:00:02/-1" : {
"basic": {
"ips": ["172.27.0.1"],
"location": "of:00000000000000b1/1"
}
}
}
"ports" : {
"of:00000000000000b1/1" : {
"interfaces" : [
{
"name" : "b1-2",
"ips" : [ "172.27.0.254/24" ],
"mac" : "fe:00:00:00:00:01"
}
]
}
$ ./vrouter.sh 172.17.0.2
onos> hosts
id=FA:00:00:00:00:01/None, mac=FA:00:00:00:00:01, location=of:00000000000000b1/25, vlan=None, ip(s)=[172.18.0.1]
id=FE:00:00:00:00:01/None, mac=FE:00:00:00:00:01, location=of:00000000000000b1/24, vlan=None, ip(s)=[172.18.0.254]
id=FE:00:00:00:00:02/None, mac=FE:00:00:00:00:02, location=of:00000000000000b1/1, vlan=None, ip(s)=[172.27.0.1], name=FE:00:00:00:00:02/None
Add route.
onos> route-add 172.27.0.0/24 172.27.0.1
onos> routes
Table: ipv4
Network Next Hop
0.0.0.0/0 172.18.0.1
172.27.0.0/24 172.27.0.1
Total: 2
Table: ipv6
Network Next Hop
Total: 0
onos> next-hops
ip=172.18.0.1, mac=FA:00:00:00:00:01, numRoutes=1
ip=172.27.0.1, mac=FE:00:00:00:00:02, numRoutes=1