Fighting at the forefront: Early 5.0.0.x experiences

 .0.0.0 came around with a ton of exciting new features I was eager to test

    • IPv6 (Dual Stack) in Underlay and Overlay 

    • Better Gateway throughput

    • Data loss prevention in SASE 

      However, new Version, new Bugs


  • I first upgraded my company Orchestrator to 5.0.0.0 (after creating a snapshot to be able to roll back to 4.5). This worked quite well, only after going to all parts of the new UI I found out that I could not reach the "General Settings" in the new UI, but the content was perfectly visible in the old UI.

    Fortunately a 5.0.0.1 upgrade solved that problem.

    Another strange items, still in version 5.0.0.1 as well, is the fact that in the old and the new UI our Edges now are showing  

    0 % memory utilization, 

    which is either an incredible efficient new code or simply a bug.

     

    Next I tried IPv6

    As my Internet provider at home still does not support IPv6, I used the new 5.0 IPv6 features to build IPv6 connectivity using the Dual Stack Overlay and the fact, that our Gateways in our Datacenter have a dual stack connectivity to the Internet.

    So I tried to ping the Google IPv6 DNS server with very mixed results:

    C:\Users\xandl>ping 2001:4860:4860::8888

     

    Pinging 2001:4860:4860::8888 with 32 bytes of data:

    Request timed out.

    Reply from 2001:4860:4860::8888: time=41ms

    Reply from 2001:4860:4860::8888: time=46ms

    Reply from 2001:4860:4860::8888: time=45ms

     

    Ping statistics for 2001:4860:4860::8888:

        Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),

    Approximate round trip times in milli-seconds:

        Minimum = 41ms, Maximum = 46ms, Average = 44ms

    The next try a Traceroute and further pings resulted in a complete loss of IPv6 connectivity:

     

    C:\Users\xandl>tracert 2001:4860:4860::8888

     Tracing route to dns.google [2001:4860:4860::8888]

    over a maximum of 30 hops:

      1  Destination host unreachable.

     Trace complete.

    C:\Users\xandl>ping 2001:4860:4860::8888

     Pinging 2001:4860:4860::8888 with 32 bytes of data:

    Destination host unreachable.

    Destination host unreachable.

    Destination host unreachable.

    Destination host unreachable.

     Ping statistics for 2001:4860:4860::8888:

        Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

    After some minutes a retry worked but only for the first request.

    So it seems that something with the Flow or the SNAT on the gateway prevents a second request to the same destination. At least until a timeout cancels the corresponding entry.

     


     But the biggest problems I faced when trying to upgrade my labenvironment to version 5.0.0.0

    Due to the fact that I overlooked the 90 days validity of the vcadmin password 2 of my 3 gateways were locked and needed a new deploy and reactivation.

    So I first upgraded the LAB Orchestrator (after a snapshot) and then redeployed the Gateways with a 5.0 image.

    But the gateways did not successfully activate even though I correctly set the necessary System parameter:

    Cloud-Init always came back with the below error

    [   83.971366] cloud-init[2986]: 2022-04-02 15:24:28,211 - cc_velocloud.py[ERROR]: Activation failed: message=b"Cannot read property 'enabled' of null\n", stderr=b'Generating RSA private key, 2048 bit long modulus (2 primes)\n..........................+++++\n.......................................................+++++\ne is 65537 (0x010001)\n'

    [   83.980086] cloud-init[2986]: 2022-04-02 15:24:28,220 - util.py[WARNING]: Running module velocloud (<module 'cloudinit.config.cc_velocloud' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_velocloud.py'>) failed

    [   84.119170] cloud-init[2986]: Cloud-init v. 21.4-0ubuntu1~18.04.1 finished at Sat, 02 Apr 2022 15:24:28 +0000. Datasource DataSourceNoCloud [seed=/dev/sr0][dsmode=net].  Up 84.08 seconds

     Even manual activation resulted in similar error message

     So I had to use my snapshot and step back to version 4.5 until this bug will be resolved.

    From friends inside VMware I heared that the bug is there whenever you try to activate or reactivate a Partner-Gateway.

     Now I also understood why in my 3rd SD-Wan environment, a SASE Proof of Concept   with Orchestrator and SASE-POP provided by VMware the version is still on 4.5, and testing new features like DLP is currently not available there.

Comments

Post a Comment

Popular posts from this blog

Orchestrator Upgrade to Version 5.2

Image
 Recently I upgraded my VMware SD-WAN lab from version 5.0 to 5.2. Since some previous versions the official documentation  recommends to ask TAC for help in upgrading.   Docu 5.2: Orchestrator Upgrade  But if you have an unsupported implementation, like I do, you need to do it on your own: Trying to copy the new version to the VCO failed and I found out that I needed to increase a disk volume How to increase disk volume on VCO is described in this blog:   VCO Upgrade to 4.5.x After increasing the disk size my copy succeeded   NOTE: I did not copy direct to the installation directory including a renaming, as once an upgrade started that file gets deleted and if there are any problems to have to redo the copy from outside.  vcadmin@vco-lab-254:~$ ls -l total 2504488 drwxr-x--- 2 root    root          4096 Aug 31  2022 20220831130007 -rw-rw---- 1 vcadmin vcadmin 2564587520 Aug 10 07:57 vco-debs-signed-5.2.0.3-R5203-20230809-GA-ff5cd1917e.tar vcadmin@vco-lab-254:~$ sudo cp vco-debs-signed
Read more

Deep Dive on DMPO and its Performance Features (available and missing) Part 1

Image
 In the last year I got quite deep insight into ther vendors SD-WAN implementation and also have seen what various vendors are critizising about the VMware DMPO features. Also there are a lot of misconceptions on the market around some of these features. So let´s try and evaluate and look a little bit deeper into typical performance features and the VMware implementation of them Features discussed in this Blog (Part 1) Per Packet Load Balancing Forward Error Correction (FEC) Packet Replication Latency Remediation The following Features will be discussed in Part 2 of my blog TCP Optimization Techniques Dejitter Buffering SAAS Application Monitoring   Per Packet Load Balancing  Unfortunately this term can be quite easily misinterpreted, when you Google for the term and get the following result:  This behaviour (often criticized by other vendors about VMware) as described above would only work quite well when you have very similar or equal connections. If for example 2 links have  both 10
Read more

Deep Dive on DMPO and its Performance Features (available and missing) Part 2

Image
  The following Features will be discussed in this second part of my blog TCP Optimization Techniques Dejitter Buffering SAAS Application Monitoring   Before diving into the mechanism here an important basic fact:     Remediation is never done for applications classified as LOW priority     TCP optimization TCP has some embedded traffic management capabilities for reliable traffic delivery  (window.size, slow-start, RTT handling,...)  But there are a bunch of factors which can negatively influence the performance, like Latency TCP-slow-start Last Mile network problems Out of Sequence packets Busty losses end host TCP limitations (missing fatures like SACK, windows scaling or timestamp options)  Typically TCP optimization helps on the transmission side and improve the download time for large data transfers over latency-high and lossy WAN links. But it can also be used for improving perfomance when on the receiver side the amount of buffers is limited and/or when the client is n
Read more