Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[mininet-discuss] More than 256 ovs switches

Gary Brown uvugdbrown at gmail.com
Wed Dec 4 00:55:25 PST 2013


I tried the dpid=xxx idea, but the datapath ID is not set to what I am
passing in.

Here is my code snippet that creates the topology and tries to set the
switches' dpid and mac address.
...
        # Host, switch, and port name formats
        self.hostFormat   = 'h%02d%02d%02d'
        self.switchFormat = 's%02d%02d%02d'
        self.ipFormat     = '%d.%d.%d.%d/%d'
        self.macFormat    = '%02x:%02x:%02x:%02x:%02x:%02x'
        self.dpidFormat   = '%02d%02d%02d0000000000'
...
        # Add hosts and switches for X, Y, and Z dimensions
        for x in range(0, self.xDim):
            for y in range(0, self.yDim):
                for z in range(0, self.zDim):

                    # Create names
                    hostName   = self.hostFormat % (x, y, z)
                    switchName = self.switchFormat % (x, y, z)

                    # Create IP and MAC addresses for host
                    ipAddress  = self.ipFormat  % (10, x, y, z, 8)
                    macAddress = self.macFormat % (0, 10, x, y, z, 0)

                    # Add host and corresponding switch
                    hostOpts = { 'ip':ipAddress, 'mac':macAddress }
                    #info( 'Creating host  : %s\n' % hostName )
                    self.addHost( hostName, **hostOpts )

                    # Add corresponding switch for host
                    dpid = self.dpidFormat % (x, y, z)
                    mac  = self.macFormat % (255, 10, x, y, z, 0)
                    info( 'Creating switch: %s (dpid=%s)\n' % (switchName,
dpid ))
                    self.addSwitch( switchName, dpid=dpid, mac=mac )
...
Here is the Mininet output along with logging by my custom topo in gray and
the dpctl show output showing the dpid in yellow, which I notice matches
the MAC address instead of the dpid I assigned.

mininet at mininet-vm:~/mininet/custom$ sudo mn --switch ovsk --ipbase=
10.0.0.0/8 --arp --custom ~/mininet/custom/topo-cray3d.py --topo
cray3d,mesh,3,3,3
X-dimension size = 3
Y-dimension size = 3
Z-dimension size = 3
Creating switch: s000000 (dpid=0000000000000000)
Creating switch: s000001 (dpid=0000010000000000)
Creating switch: s000002 (dpid=0000020000000000)
Creating switch: s000100 (dpid=0001000000000000)
Creating switch: s000101 (dpid=0001010000000000)
Creating switch: s000102 (dpid=0001020000000000)
Creating switch: s000200 (dpid=0002000000000000)
Creating switch: s000201 (dpid=0002010000000000)
Creating switch: s000202 (dpid=0002020000000000)
Creating switch: s010000 (dpid=0100000000000000)
Creating switch: s010001 (dpid=0100010000000000)
Creating switch: s010002 (dpid=0100020000000000)
Creating switch: s010100 (dpid=0101000000000000)
Creating switch: s010101 (dpid=0101010000000000)
Creating switch: s010102 (dpid=0101020000000000)
Creating switch: s010200 (dpid=0102000000000000)
Creating switch: s010201 (dpid=0102010000000000)
Creating switch: s010202 (dpid=0102020000000000)
Creating switch: s020000 (dpid=0200000000000000)
Creating switch: s020001 (dpid=0200010000000000)
Creating switch: s020002 (dpid=0200020000000000)
Creating switch: s020100 (dpid=0201000000000000)
Creating switch: s020101 (dpid=0201010000000000)
Creating switch: s020102 (dpid=0201020000000000)
Creating switch: s020200 (dpid=0202000000000000)
Creating switch: s020201 (dpid=0202010000000000)
Creating switch: s020202 (dpid=0202020000000000)
*** Creating network
*** Adding controller
*** Adding hosts:
h000000 h000001 h000002 h000100 h000101 h000102 h000200 h000201 h000202
h010000 h010001 h010002 h010100 h010101 h010102 h010200 h010201 h010202
h020000 h020001 h020002 h020100 h020101 h020102 h020200 h020201 h020202
*** Adding switches:
s000000 s000001 s000002 s000100 s000101 s000102 s000200 s000201 s000202
s010000 s010001 s010002 s010100 s010101 s010102 s010200 s010201 s010202
s020000 s020001 s020002 s020100 s020101 s020102 s020200 s020201 s020202
*** Adding links:
(1.60Mbit) (1.60Mbit) (h000000, s000000) (1.60Mbit) (1.60Mbit) (h000001,
s000001) (1.60Mbit) (1.60Mbit) (h000002, s000002) (1.60Mbit) (1.60Mbit)
(h000100, s000100) (1.60Mbit) (1.60Mbit) (h000101, s000101) (1.60Mbit)
(1.60Mbit) (h000102, s000102) (1.60Mbit) (1.60Mbit) (h000200, s000200)
(1.60Mbit) (1.60Mbit) (h000201, s000201) (1.60Mbit) (1.60Mbit) (h000202,
s000202) (1.60Mbit) (1.60Mbit) (h010000, s010000) (1.60Mbit) (1.60Mbit)
(h010001, s010001) (1.60Mbit) (1.60Mbit) (h010002, s010002) (1.60Mbit)
(1.60Mbit) (h010100, s010100) (1.60Mbit) (1.60Mbit) (h010101, s010101)
(1.60Mbit) (1.60Mbit) (h010102, s010102) (1.60Mbit) (1.60Mbit) (h010200,
s010200) (1.60Mbit) (1.60Mbit) (h010201, s010201) (1.60Mbit) (1.60Mbit)
(h010202, s010202) (1.60Mbit) (1.60Mbit) (h020000, s020000) (1.60Mbit)
(1.60Mbit) (h020001, s020001) (1.60Mbit) (1.60Mbit) (h020002, s020002)
(1.60Mbit) (1.60Mbit) (h020100, s020100) (1.60Mbit) (1.60Mbit) (h020101,
s020101) (1.60Mbit) (1.60Mbit) (h020102, s020102) (1.60Mbit) (1.60Mbit)
(h020200, s020200) (1.60Mbit) (1.60Mbit) (h020201, s020201) (1.60Mbit)
(1.60Mbit) (h020202, s020202) (0.75Mbit) (0.75Mbit) (s000000, s000001)
(0.75Mbit) (0.75Mbit) (s000000, s000100) (0.75Mbit) (0.75Mbit) (s000000,
s010000) (0.75Mbit) (0.75Mbit) (s000001, s000002) (0.75Mbit) (0.75Mbit)
(s000001, s000101) (0.75Mbit) (0.75Mbit) (s000001, s010001) (0.75Mbit)
(0.75Mbit) (s000002, s000102) (0.75Mbit) (0.75Mbit) (s000002, s010002)
(0.75Mbit) (0.75Mbit) (s000100, s000101) (0.38Mbit) (0.38Mbit) (s000100,
s000200) (0.75Mbit) (0.75Mbit) (s000100, s010100) (0.75Mbit) (0.75Mbit)
(s000101, s000102) (0.38Mbit) (0.38Mbit) (s000101, s000201) (0.75Mbit)
(0.75Mbit) (s000101, s010101) (0.38Mbit) (0.38Mbit) (s000102, s000202)
(0.75Mbit) (0.75Mbit) (s000102, s010102) (0.75Mbit) (0.75Mbit) (s000200,
s000201) (0.75Mbit) (0.75Mbit) (s000200, s010200) (0.75Mbit) (0.75Mbit)
(s000201, s000202) (0.75Mbit) (0.75Mbit) (s000201, s010201) (0.75Mbit)
(0.75Mbit) (s000202, s010202) (0.75Mbit) (0.75Mbit) (s010000, s010001)
(0.75Mbit) (0.75Mbit) (s010000, s010100) (0.75Mbit) (0.75Mbit) (s010000,
s020000) (0.75Mbit) (0.75Mbit) (s010001, s010002) (0.75Mbit)
(0.75Mbit) (s010001,
s010101) (0.75Mbit) (0.75Mbit) (s010001, s020001) (0.75Mbit) (0.75Mbit)
(s010002, s010102) (0.75Mbit) (0.75Mbit) (s010002, s020002) (0.75Mbit)
(0.75Mbit) (s010100, s010101) (0.38Mbit) (0.38Mbit) (s010100, s010200)
(0.75Mbit) (0.75Mbit) (s010100, s020100) (0.75Mbit) (0.75Mbit) (s010101,
s010102) (0.38Mbit) (0.38Mbit) (s010101, s010201) (0.75Mbit)
(0.75Mbit) (s010101,
s020101) (0.38Mbit) (0.38Mbit) (s010102, s010202) (0.75Mbit) (0.75Mbit)
(s010102, s020102) (0.75Mbit) (0.75Mbit) (s010200, s010201) (0.75Mbit)
(0.75Mbit) (s010200, s020200) (0.75Mbit) (0.75Mbit) (s010201, s010202)
(0.75Mbit) (0.75Mbit) (s010201, s020201) (0.75Mbit) (0.75Mbit) (s010202,
s020202) (0.75Mbit) (0.75Mbit) (s020000, s020001) (0.75Mbit) (0.75Mbit)
(s020000, s020100) (0.75Mbit) (0.75Mbit) (s020001, s020002) (0.75Mbit)
(0.75Mbit) (s020001, s020101) (0.75Mbit) (0.75Mbit) (s020002, s020102)
(0.75Mbit) (0.75Mbit) (s020100, s020101) (0.38Mbit) (0.38Mbit) (s020100,
s020200) (0.75Mbit) (0.75Mbit) (s020101, s020102) (0.38Mbit) (0.38Mbit)
(s020101, s020201) (0.38Mbit) (0.38Mbit) (s020102, s020202) (0.75Mbit)
(0.75Mbit) (s020200, s020201) (0.75Mbit) (0.75Mbit) (s020201, s020202)
*** Configuring hosts
h000000 h000001 h000002 h000100 h000101 h000102 h000200 h000201 h000202
h010000 h010001 h010002 h010100 h010101 h010102 h010200 h010201 h010202
h020000 h020001 h020002 h020100 h020101 h020102 h020200 h020201 h020202
*** Starting controller
*** Starting 27 switches
s000000 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit) s000001 (1.60Mbit)
(0.75Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit) s000002 (1.60Mbit) (0.75Mbit)
(0.75Mbit) (0.75Mbit) s000100 (1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit)
(0.75Mbit) s000101 (1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit)
(0.75Mbit) s000102 (1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit)
s000200 (1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) s000201 (1.60Mbit)
(0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit) s000202 (1.60Mbit) (0.75Mbit)
(0.38Mbit) (0.75Mbit) s010000 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit)
(0.75Mbit) s010001 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit)
(0.75Mbit) s010002 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit)
s010100 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit)
(0.75Mbit) s010101
(1.60Mbit) (0.75Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit)
(0.75Mbit)s010102 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.38Mbit)
(0.75Mbit) (0.75Mbit)
s010200 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) s010201
(1.60Mbit) (0.75Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit) s010202
(1.60Mbit) (0.75Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) s020000 (1.60Mbit)
(0.75Mbit) (0.75Mbit) (0.75Mbit) s020001 (1.60Mbit) (0.75Mbit) (0.75Mbit)
(0.75Mbit) (0.75Mbit) s020002 (1.60Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit)
s020100 (1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit) s020101
(1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit) (0.75Mbit) s020102
(1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit) (0.75Mbit) s020200 (1.60Mbit)
(0.75Mbit) (0.38Mbit) (0.75Mbit) s020201 (1.60Mbit) (0.75Mbit) (0.38Mbit)
(0.75Mbit) (0.75Mbit) s020202 (1.60Mbit) (0.75Mbit) (0.38Mbit) (0.75Mbit)
*** Starting CLI:
mininet> s010101 dpctl show tcp:127.0.0.1:6634
features_reply (xid=0x86e66066): ver:0x1, dpid:5a722ed54341
n_tables:255, n_buffers:256
features: capabilities:0xc7, actions:0xfff
 1(s000000-eth1): addr:6e:56:e2:6a:bd:89, config: 0, state:0
     current:    10GB-FD COPPER
 2(s000000-eth2): addr:b2:ad:bc:96:3e:08, config: 0, state:0
     current:    10GB-FD COPPER
 3(s000000-eth4): addr:d6:86:49:e7:6b:9f, config: 0, state:0
     current:    10GB-FD COPPER
 4(s000000-eth6): addr:3e:54:bd:1f:84:6d, config: 0, state:0
     current:    10GB-FD COPPER
 LOCAL(s000000): addr:5a:72:2e:d5:43:41, config: 0x1, state:0x1
get_config_reply (xid=0x11accab2): miss_send_len=0
mininet>

I have also noticed something quite odd, which is switch s010101 should
have 7 ports, eth1-eth7, but dpctl shows only 4 while Mininet created 7
ports (highlighted in red) for 7 links and started them (also in red).  Any
idea why the discrepancy?
Thanks for any explanations you can give.




On Tue, Dec 3, 2013 at 11:35 PM, Gary Brown <uvugdbrown at gmail.com> wrote:

> Thank you for your reply, Bob.
> I did not know about the ability to set the datapath id, which is what I
> will do.  I was also expecting to do my own controller, but needed a way to
> detect which switch I was dealing with, and your datapath id answer will
> fit the bill perfectly, and answers my question I submitted a few minutes
> ago about the correlation between switch name and datapath id (none, unless
> one makes it so by setting the datapath id).
> Thanks for the answers!
>
> Gary D. Brown
>
>
>
> On Tue, Dec 3, 2013 at 11:21 PM, Bob Lantz <rlantz at cs.stanford.edu> wrote:
>
>> The default controller will certainly not work, as it implements an
>> Ethernet bridge without STP (which you don't want anyway since it would
>> remove most of your links!) I expect you will need to create your own
>> controller. Which should be relatively straightforward and fun for a torus
>> topology.
>>
>> I also recommend starting with a very small topology, like 4x4 rather
>> than 16x16, and getting your controller working with that topology first.
>>
>> Regarding dpid, you can probably set it yourself using something like
>> addSwitch( 's%d' % num, dpid=('%016x' % num ) )
>>
>> Depending on your kernel, you may also wish to deactivate IPv6 in each of
>> your mininet hosts to avoid lots of neighbor discovery packets.
>>
>>
>> On Dec 3, 2013, at 9:45 PM, Gary Brown <uvugdbrown at gmail.com> wrote:
>>
>> Thank you to everyone who replied to me privately.
>> I have not run a controller, so I assume the default controller is
>> operational.
>> So far, all I had done was define a 3D torus topology in Mininet using a
>> topo module I wrote, which worked fine for a smaller network as long as the
>> quantity of vSwitches does not seem to exceed 128-132 or so.
>> However, when running Mininet from a putty window and while it was
>> setting the ARP table for the hosts (one host per vSwitch) to prevent an
>> ARP storm, my putty windows (I had two open) would get a "PuTTY Fatal
>> Error" message of "Network error: Software caused connection abort".  At
>> that point they were completely non-responsive and I had to close them.
>> When I tried to reconnect with a new putty window, I got a "Network
>> error: Connection timed out" error and could only use the VM console.
>> Through the console I later found Mininet (mn) was still running along
>> with 576 "bash" processes (one per 288 hosts and 288 vSwitches) and
>> according to top, ovs-vswitchd was consuming ~75% of the CPU and
>> ovsdb-server was consuming the remaining 25%, which probably explained the
>> putty windows failing.
>> I found when I ran Mininet in the VM console window, that I had no
>> problem creating many switches.  It appears when creating the static ARP
>> tables after creating the hosts and then creating the switches and their
>> links that the OVS daemons mentioned above basically kill the VM's
>> performance and I lose the non-console windows, which was not apparent
>> until I attempted to hit a key (e.g. ctrl-c) in a window and it timed out.
>> Mininet is creating the large number of switches I wanted; I just cannot
>> do it from any window excepts the VM's console window as the other windows
>> do not get any CPU cycles in a timely manner.
>> After discovering this problem and finding I have to use the VM's
>> console, I have been able to create large numbers of switches.  To use
>> other windows to the VM, I just have to wait until after Mininet has
>> finished creating the hosts and switches and linked everything up before I
>> can use non-console windows successfully.  The fact the windows became
>> unresponsive had appeared to me as a Mininet failure, when in fact, it had
>> not failed at all, just the windows had.
>> Well, just another odd set of facts learned!
>>
>> --
>> Gary D. Brown
>>
>>
>> On Tue, Dec 3, 2013 at 8:10 PM, Brian O'Connor <bocon at cs.stanford.edu>wrote:
>>
>>> Sorry, fixLimits is actually called the first time a Mininet object is
>>> instantiated (either using "mn" or in a custom script).
>>>
>>> If you are using a custom script and you are using a Mininet object, you
>>> should be able to create 500+ switches (although it might take awhile).
>>>
>>> If you are using the lower level API (creating Host and Link objects
>>> directly), then you may need to call fixLimits to limit the file descriptor
>>> limit.
>>>
>>> - Brian
>>>
>>>
>>> On Tue, Dec 3, 2013 at 7:04 PM, Brian O'Connor <bocon at cs.stanford.edu>wrote:
>>>
>>>> Hi Gary,
>>>>
>>>> The limit appears to be related to the file descriptor limit.
>>>>
>>>> When you invoke "mn" from the command line, we run util.fixLimits to
>>>> increase the number of allowed file descriptors.
>>>>
>>>> If you are running a custom Python script, you can try:
>>>>
>>>> from mininet.util import fixLimits
>>>> fixLimits()
>>>>
>>>> - Brian
>>>>
>>>>
>>>>
>>>> On Thu, Nov 28, 2013 at 7:19 AM, Gary Brown <uvugdbrown at gmail.com>wrote:
>>>>
>>>>> I am trying to recreate an actual network topology experiment I did
>>>>> this past summer that requires 288 vSwitches, but when I use Mininet to
>>>>> create the network, I get an exception at about 256 switches (highly
>>>>> suspicious) and Mininet cleans everything up and quits.
>>>>> I have seen posts on this mailing list indicating people are creating
>>>>> experiments with more than 256 switches, up to 1024 and even 1200.  My
>>>>> online searching has indicated ovs switches have a limit of 256 switches
>>>>> (see Limits at bottom of webpage at
>>>>> http://openvswitch.org/cgi-bin/ovsman.cgi?page=vswitchd%2Fovs-vswitchd.8
>>>>> ).
>>>>> Has anyone been able to create more than 256 ovs switches and if so,
>>>>> how did you do so, or at least how did you raise the limit?
>>>>> I am using the standard Mininet 2.1.0 VM running Ubuntu 13.04 inside
>>>>> VirtualBox 4.2.18 r88780 on a Windows 7 laptop with Core i7 (Ivy Bridge)
>>>>> and 32 GB of memory.
>>>>>
>>>>> --
>>>>> Gary
>>>>> M.S. student, University of Utah
>>>>>
>>>>> _______________________________________________
>>>>> mininet-discuss mailing list
>>>>> mininet-discuss at lists.stanford.edu
>>>>> https://mailman.stanford.edu/mailman/listinfo/mininet-discuss
>>>>>
>>>>>
>> _______________________________________________
>> mininet-discuss mailing list
>> mininet-discuss at lists.stanford.edu
>> https://mailman.stanford.edu/mailman/listinfo/mininet-discuss
>>
>>
>>
>
>
> --
> Gary D. Brown
>



-- 
Gary D. Brown
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/mininet-discuss/attachments/20131204/94e2276e/attachment-0001.html>


More information about the mininet-discuss mailing list