MIKROTIK + ROUTEROS - FAELIX · PDF fileMIKROTIK + ROUTEROS MIKROTIK IS BIG IN WISPs (though...

Marek Isalski – marek @ faelix.net – @maznu faelix limited – https://faelix.net/ – @faelix

PDF: https://faelix.link/netmcr7 (8Mb)

MIKROTIK + ROUTEROS

2500+ PEOPLEMUM INDONESIA 2015

MIKROTIK + ROUTEROS

MIKROTIK IS BIG IN…

▸ WISPs (though Ubiquiti is very popular in UK/US too)

▸ Mali (rural Internet infrastructure)

▸ …Burkina Faso, Brazil, Czech Republic, Hungary…

▸ Uruguay (under OLPC programme)

▸ …bit of a cult following in UK?

MIKROTIK + ROUTEROS

INTRODUCTIONS

▸ MikroTik = company ("MikroTik SIA")Established 1996 in Latvia 180+ employees

▸ Mikro = smallTik = network

▸ RouterOS = Linux kernel + routing protocols + other stuff v6.38 is current as of today

▸ RouterBOARD = hardware First one made in 2002

€

MIKROTIK + ROUTEROS

ROUTEROS: VERSIONS 6 AND 7

▸ v6.00 — 2013-05-20 — …and roughly monthly until…v6.33 — 2015-11-06 — "long term" support of point versions v6.34 — 2016-01-29 — CHR v6.35 — 2016-04-26 — LTEv6.36 — 2016-07-21 — certificates, IPsec, bugs + fixes v6.37 — 2016-09-23 — CAPsMANv2 v6.38 — 2016-01-02 — IKEv2

▸ v7.00 — ????-??-??

TEXT

FEATURES

▸ OOB/management: telnet, ssh, http(s), API(ssl), FTP, RS232, USB

▸ Linux kernel, IPv4 + IPv6 forwarding, ip(6)tables, bridges, queues

▸ Virtual: VLAN, bonding, OpenVPN, L2TP (LNS/LAC), SSTP, IPsec, IKEv2, GRE, EoIP, MPLS/VPLS, VRRP…

▸ Packet steering: BFD, RIP(ng), BGP, OSPF(v3), MME, OpenFlow.

▸ Also: DHCP(v6), DNS, SMB, SNMP, TFTP, HTTP Proxy, mtr, traffic generator, bandwidth test, ping, torch, The Dude, user-man, NTP, RS232 console, captive portal…

MIKROTIK + ROUTEROS

RELAX: IT'S JUST LINUX!

MIKROTIK + ROUTEROS

RELAX: IT'S JUST LINUX!

MIKROTIK + ROUTEROS

RELAX: IT'S JUST LINUX!

MIKROTIK + ROUTEROS

RELAX: IT'S JUST LINUX!

MPLS on Linux!

MIKROTIK + ROUTEROS

HARDWARE

▸ MIPS, SMIPS, MMIPS, PPC, ARM, Tile, x86, x64, virtual machine

▸ 100M/1G/10G ethernet (various common vendors) RJ45, SFP, SFP+ (miniGBIC) formats

▸ 802.11 b/g/n, a/n, ac (Atheros chipsets only?)

▸ LTE (USB dongle? check it's supported!)

MIKROTIK + ROUTEROS

LICENSING

▸ Hardware comes with never expiring license.

▸ 0 = trial (24 hours only)1 = free demo (limited to one of anything)

▸ 3 = WISP CPE (limits on some interface types, BGP; not an AP) 4 = WISP (can be an AP; but limits on some interface types)

▸ 5 = "router" (basically good for hundreds of users) 6 = Controller (unlimited everything)

"GPL VIOLATIONS!"

mailing lists, etc

MIKROTIK + ROUTEROS

CONTROVERSY!

MIKROTIK + ROUTEROS

LICENSING

▸ Object code comes with hardware. You pay for hardware.

▸ GPL says source should be as easy to get as object code.

▸ MikroTik seemed to think this meant, "so you can send $45 to us to send you a CD with source code too!"

▸ Following the word but not the spirit?

▸ Email and ask for patches, they are forthcoming:e.g. https://dev.openwrt.org/ticket/4948

"MIKROTIKS ARE THE BREXIT OF ROUTERS!"

UKNOT passim

MIKROTIK + ROUTEROS

CONTROVERSY!

"THEY'RE BEING PWNED!"

Brian Krebs

MIKROTIK + ROUTEROS

CONTROVERSY!

Marek Isalski

MIKROTIK + ROUTEROS

MIKROTIK + ROUTEROS

WIRELESS: LONGHAUL

LHG

SXTmANT

LDF

833MBIT/S ~£100

MIKROTIK + ROUTEROS

WIRELESS: INDOOR

wAP

mAP

hAP

5-60V ~£20

MIKROTIK + ROUTEROS

BARE "ROUTERBOARD"

RB922 RB800

MIKROTIK + ROUTEROS

BARE "ROUTERBOARD"

MIKROTIK + ROUTEROS

CPE GEAR

hEXRB2011

RB3011

1GBIT/SEC ~£50

MIKROTIK + ROUTEROS

BIG TOYS CRS125 + CRS226

101610361072

100MPPS £3000

CCR 100910GE £300

MIKROTIK + ROUTEROS

"THE CLOUD"

▸ Cloud-Hosted Router (CHR) is x86/x64 VM imageAWS-ready image; Azure works; we run under Xen; maybe KVM?

▸ $0 = 1Mbit/sec/interface$45 = 1Gbit/sec/interface$95 = 10Gbit/sec/interface$250 = ∞/interface

▸ As many virtual ethernet interfaces as you like!

▸ Evaluation, upgrade test, labs, education, interop, VPN endpoints, wireless controllers, "cloud"…

MIKROTIK + ROUTEROS

COMMAND-LINE FTW!

▸ /ip address add interface=ether1 address=192.168.88.1/24

▸ /ip route add dst-address=8.8.8.8/32 gateway=192.168.88.2print where dst-address=8.8.8.8/32

▸ /ping 8.8.8.8

▸ /ip route export

MIKROTIK + ROUTEROS

WANT A VLAN?

▸ /interface vlanadd interface=ether1 name=ether1-vlan1000 vlan-id=1000

▸ /ip address add interface=ether1-vlan1000 address=192.168.88.1/24

MIKROTIK + ROUTEROS

WANT A LOOPBACK?

▸ /interface bridgeadd name=loopy protocol-mode=none

▸ /ip address add interface=loopy address=127.0.0.42/32

MIKROTIK + ROUTEROS

WANT BONDING/TRUNKING/ETHERCHANNEL/AGG…?

▸ /interface bondingadd name=bondy mode=active-backup primary=ether1 slaves=ether1,ether2

▸ /ip address add interface=bondy address=203.0.113.1/24

MIKROTIK + ROUTEROS

WANT 1500 MTU LAYER-2 USING ADSL BACKHAUL?

▸ /interface eoipadd name=tunnel clamp-tcp-mss=no mtu=1500 tunnel-id=1local-address=203.0.113.1 remote-address=198.51.100.1

▸ /ip address add interface=tunnel address=192.168.88.1/24

▸ /interface eoipadd name=tunnel clamp-tcp-mss=no mtu=1500 tunnel-id=1local-address=198.51.100.1 remote-address=203.0.113.1

▸ /ip address add interface=tunnel address=192.168.88.2/24

MIKROTIK + ROUTEROS

LINE OF SIGHT AKA BABY WISP

▸ /interface wireless set mode=bridge frequency=2412 band=2ghz-b/g/n channel-width=20/40mhz-Ce ssid=wispysecurity-profile=babywisp wireless-protocol=802.11

▸ /interface wireless security-profiles add name=babywisp authentication-types=wpa2-psk mode=dynamic-keyswpa2-pre-shared-key=donttellanyonethepassword

▸ /interface wireless set mode=station-bridge frequency=2412 band=2ghz-b/g/n channel-width=20/40mhz-Ce ssid=wispysecurity-profile=babywisp wireless-protocol=802.11

MIKROTIK + ROUTEROS

LINE OF SIGHT AKA BABY WISP

MIKROTIK + ROUTEROS

LINE OF SIGHT AKA WARDRIVING

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

▸ Centralise AP management

▸ All SSIDs, VLANs, brought back to the controller

▸ £20-130 per AP£50-3000 for controller

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

MIKROTIK + ROUTEROS

WIRELESS DEPLOYMENT

MIKROTIK + ROUTEROS

BUDGET PROVIDER EDGE

▸ 2x £300 CCR1009 — 15Gbit/sec or 15Mpps 2x £250 CRS226 — 88Gbit/sec or 64Mpps3x copper SFP+

108 watts!"ISP" for <£1200(just add servers)

MIKROTIK + ROUTEROS

BUDGET PROVIDER EDGE

▸ /routing bgp instanceset default as=41495 client-to-client-reflection=norouter-id=192.0.2.1

▸ /routing bgp network add network=198.51.100.0/24

▸ /routing bgp peeradd name=AS174.v4.gw remote-as=174 in-filter=v4-i-AS174 out-filter=v4-o-upstream remote-address=203.0.113.174

▸ /routing bgp peeradd name=AS174.v6.gw remote-as=174 address-families=ipv6 in-filter=v6-i-AS174 out-filter=v6-o-AS174 remote-address=…

BCP38

MIKROTIK + ROUTEROS

ROUTEROS SWITCHES AND VLANS

MIKROTIK + ROUTEROS

ROUTEROS SWITCHES AND VLANS

▸ interface ethernet 1 untagged 1000 tagged 1001-1099

▸ interface ethernet 2 untagged 1000 tagged 1001-1099

MIKROTIK + ROUTEROS

ROUTEROS SWITCHES AND VLANS

▸ interface FastEthernet0/1 switchport mode trunk switchport trunk native vlan 1000 switchport allowed vlan 1001,1002,1003,…1099

▸ interface FastEthernet0/2 switchport mode trunk switchport trunk native vlan 1000 switchport allowed vlan 1001,1002,1003,…1099

MIKROTIK + ROUTEROS

ROUTEROS SWITCHES AND VLANS

▸ /interface ethernet switch set drop-if-invalid-or-src-port-not-member-of-vlan-on-ports=ether01,ether02,… drop-if-no-vlan-assignment-on-ports=ether01,ether02,…

▸ /interface ethernet switch egress-vlan-tag add tagged-ports="ether01,ether02,…" vlan-id=1001 add tagged-ports="ether01,ether02,…" vlan-id=1002 …

▸ /interface ether switch ingress-vlan-translation add customer-vid=0 customer-vlan-format=untagged-or-tagged new-customer-vid=1000 ports="ether01,ether02,…"

▸ /interface ethernet switch vlanadd ports="ether01,ether02,…" vlan-id=1000 add ports="ether01,ether02,…" vlan-id=1001 …

MIKROTIK + ROUTEROS

ROUTEROS SWITCHES AND VLANS

▸ /interface ethernet switch set drop-if-invalid-or-src-port-not-member-of-vlan-on-ports=ether01,ether02,… drop-if-no-vlan-assignment-on-ports=ether01,ether02,…

▸ /interface ethernet switch egress-vlan-tag add tagged-ports="ether01,ether02,…" vlan-id=1001 add tagged-ports="ether01,ether02,…" vlan-id=1002 …

▸ /interface ether switch ingress-vlan-translation add customer-vid=0 customer-vlan-format=untagged-or-tagged new-customer-vid=1000 ports="ether01,ether02,…"

▸ /interface ethernet switch vlanadd ports="ether01,ether02,…" vlan-id=1000 add ports="ether01,ether02,…" vlan-id=1001 …

D:

MIKROTIK + ROUTEROS

OVERALL EXPERIENCE

▸ Some weird behaviour occasionally…

▸ Disable VLAN interface before changing its physical interface or VID

▸ Support are helpful and fast; anecdotally, as responsive as the "big name" vendors

▸ Debugging time = get friendly with RouterOS command-line

MIKROTIK + ROUTEROS

THE GOOD THE BAD

▸ £700 + 70W routes >10Gbit/s

▸ BGP feels familiar after years of experience of Quagga

▸ Consultants out there if you need them; training & quals

▸ MikroTik now "go to" choice for CPE, wireless, etc…

▸ Vendor interop good (beware of extra options in RouterOS)

▸ BGP converge & FIB is slow on CCR with 2M+ routes

▸ Routing filters don't always work first time (enable/disable)

▸ IPv6 BGP recursive nexthop

▸ Switch VLAN setup feels like raw config of merchant silicon

▸ "RouterOS 7"

e: marek@faelix.net t: @maznu w: https://faelix.net/

THANKS FOR LISTENING!

ANY QUESTIONS?