OVN IPSec #4

fdobrovolny · 2024-05-08T07:00:24Z

A prerequisite to this PR is a custom-build ipsec image: SovereignCloudStack/kolla#1 (find it in the SCS registry registry.scs.community/kolla/openvswitch-ipsec:18.1.0)

To test this PR you can do the following:

Enable OVN as the networking plugin:
In globals.yml set
neutron_plugin_agent: "ovn"
Enable OVN IPSEC with certificate generation

enable_ovn_ipsec: true
neutron_ovs_generate_certificates: true
For this PR a test image is currently available at:

openvswitch_ipsec_image_full: "registry.scs.community/kolla/openvswitch-ipsec:18.1.0"

This change automates the prometheus blackbox monitoring configuration for common endpoints. Custom endpoints can be added to prometheus_blackbox_exporter_endpoints_custom. Change-Id: Id6f51a2bebee3ab63b84ca7032aad17c2933838c

fdobrovolny · 2024-06-05T14:43:30Z

A prerequisite to this PR is a custom-build ipsec image:
fdobrovolny/kolla#1

To test this PR you can do the following:

Enable OVN as the networking plugin:
In globals.yml set neutron_plugin_agent: "ovn"
Enable OVN IPSEC with certificate generation

enable_ovn_ipsec: true
neutron_ovs_generate_certificates: true

For this PR a test image is currently available at:

openvswitch_ipsec_image_full: "registry.scs.community/kolla/openvswitch-ipsec:17.1.0"

Currently, the process_monitor in the supervisord is broken and not working.

ansible/roles/openvswitch/templates/openvswitch-ipsec-supervisor.conf.j2

ansible/roles/openvswitch/templates/start-ovs-ipsec.j2

ansible/roles/openvswitch/templates/start-ipsec.j2

ansible/roles/openvswitch/files/ovn_ipsec_supervisor_event_handler.py

ansible/roles/certificates/tasks/generate-ovs.yml

ansible/roles/certificates/tasks/main.yml

ansible/roles/certificates/templates/openssl-kolla-ovs.cnf.j2

ansible/roles/ovn-db/tasks/bootstrap-db.yml

This still does not work, however; for those coming after me, I'm pushing this, so you don't have to spend time on this version.

This should now avoid issues when restarting IPSEC

this is now more in line with other retry mechanisms: use a local variable, use 10 attempts, sleep 10 seconds. reason: there are still occasional resize failures in CI. Signed-off-by: Sven Kieske <[email protected]> Change-Id: I2ebb17459043ce677e72528bc7490b5728af396a

Change-Id: Ic28c945a1dda69813073eda8937b977ae97294ac

Copied files are no longer needed for the s6-overlay based supervision and also command is set up empty as s6-overlay is managing processes in the container Change-Id: Iefe93965351b5adc8285042b3012a60ee4853140

Change-Id: I6599761a68924b97bf62e32e66325f30b8b09c06

matofeder

I tested the change with a multinode OpenStack instance, and it works!

Steps to reproduce my test setup:

Set the following variables in globals.yml:

neutron_plugin_agent: "ovn"
enable_ovn_ipsec: true
neutron_ovs_generate_certificates: true
openvswitch_ipsec_image_full: "registry.scs.community/kolla/openvswitch-ipsec:18.1.0"

Deploy OpenStack by kolla-ansible, multinode setup with 3 compute nodes
Spin up 3 VMs on top of OpenStack, and verify that each runs on a separate hypervisor (compute node)

**$ openstack server list -c Name -c Host -c Networks
+-------+-------------------------------------+----------------------+
| Name  | Networks                            | Host                 |
+-------+-------------------------------------+----------------------+
| demo3 | demo-net=10.0.0.75, 192.168.124.87  | openstack-compute-02 |
| demo2 | demo-net=10.0.0.237, 192.168.124.70 | openstack-compute-01 |
| demo1 | demo-net=10.0.0.169, 192.168.124.30 | openstack-compute-03 |
+-------+-------------------------------------+----------------------+
**

Send some messages between VMs and try to simulate a man-in-the-middle attack with tcpdump listening to all TCP packets on one hypervisor node.

Test result:

I was not able to read messages, as they are encrypted by IPSec (i see some ESP packets there)
OVS does contain some IPsec connections, see

**kolla@openstack-compute-02:~$ docker exec  -it openvswitch_ipsec bash
(openvswitch-ipsec)[root@openstack-compute-02 /]#                                                        
(openvswitch-ipsec)[root@openstack-compute-02 /]#                                                        
(openvswitch-ipsec)[root@openstack-compute-02 /]# ovs-appctl -t ovs-monitor-ipsec tunnels/show           
Interface name: ovn-openst-0 v1 (CONFIGURED)                                                             
  Tunnel Type:    geneve                                                                                 
  Local IP:       192.168.123.82                                                                         
  Remote IP:      192.168.123.132                                                                        
  Address Family: IPv4                                                                                   
  SKB mark:       None                                                                                                                                                                                             
  Local cert:     /etc/openvswitch/cert.pem                                                                                                                                                                        
  Local name:     openstack-compute-02                                                                   
  Local key:      /etc/openvswitch/key.pem                                                               
  Remote cert:    None                                                                                   
  Remote name:    openstack-compute-03                                                                   
  CA cert:        /etc/openvswitch/root.pem                                                              
  PSK:            None                                                                                   
  Custom Options: {}                                                                                     
  Ofport:         1                                                                                      
  CFM state:      Disabled                                                                               
Kernel policies installed:                                                                               
  src 192.168.123.82/32 dst 192.168.123.132/32 proto udp dport 6081                                      
  src 192.168.123.82/32 dst 192.168.123.132/32 proto udp dport 6081                                      
  src 192.168.123.82/32 dst 192.168.123.132/32 proto udp sport 6081                                      
  src 192.168.123.82/32 dst 192.168.123.132/32 proto udp sport 6081                                      
Kernel security associations installed:                                                                  
  sel src 192.168.123.132/32 dst 192.168.123.82/32 proto udp sport 6081                                  
  sel src 192.168.123.82/32 dst 192.168.123.132/32 proto udp dport 6081                                  
  sel src 192.168.123.132/32 dst 192.168.123.82/32 proto udp dport 6081                                  
  sel src 192.168.123.82/32 dst 192.168.123.132/32 proto udp sport 6081
IPsec connections that are active:                                                                       
  000 #31: "ovn-openst-0-in-1" [email protected] [email protected] Traffic: ESPin=132B ESPout=0B ESPmax=2^63B                                                                                 
  000 #35: "ovn-openst-0-out-1":500 STATE_V2_ESTABLISHED_IKE_SA (established IKE SA); REKEY in 6869s; REPLACE in 7139s; newest; idle; **
...

Edit:
I also tested the case when the 2 VMs communicate within one hypervisor host. It this case the communication between VMs is not encrypted which is intended.

Overall LGTM, I tested the functionality of the PoC code and it works as expected! I did not look into the code itself so much.

matofeder · 2024-09-25T12:18:39Z

ansible/roles/openvswitch/templates/openvswitch-ipsec-supervisor.conf.j2

@@ -0,0 +1,35 @@
+[supervisord]
+nodaemon=true


I guess this and all supervisor related things could be removed

matofeder · 2024-09-25T12:19:46Z

ansible/roles/ironic/defaults/main.yml

@@ -253,7 +253,7 @@ ironic_inspector_default_volumes:
  - "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}"
  - "kolla_logs:/var/log/kolla"
  - "ironic_inspector_dhcp_hosts:/var/lib/ironic-inspector/dhcp-hostsdir"
-  - "{{ kolla_dev_repos_directory ~ '/ironic-inspector/ironic_inspector:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/ironic_inspector' if ironic_inspector_dev_mode | bool else '' }}"
+  - "{{ kolla_dev_repos_directory ~ '/ironic-inspector:/dev-mode/ironic-inspector' if ironic_inspector_dev_mode | bool else '' }}"


Could we remove this dev-mode change from all OS services?

matofeder · 2024-09-25T12:21:34Z

ansible/roles/ironic/templates/ironic-inspector.conf.j2

+{% if ironic_inspector_known_devices_file_path is defined %}
+[accelerators]
+known_devices = /etc/ironic-inspector/known_devices.yaml
+{% endif %}


I suspect that this change does not relate to the IPsec and comes from upstream,
Be sure that we clean this branch (rebase) on the latest upstream before we push it upstream.

Change-Id: I88da70757ae3a9bb2c3a506a064e4f1b7193b6c0

Kolla-ansible itself requires ansible-core>=2.16,<2.18, but ansible-core in this version no longer supports python38 and python39 as per [1]. So let's just drop this old python support. [1] https://github.com/ansible/ansible/blob/v2.16.11/setup.cfg Change-Id: Ic8aaa57f75479a17c215c27ac5e6df0f18c74edc

Removing scripts and their copy tasks which aren't used in s6-overlay solution. Change-Id: Ieec19d3f9095a23358ad8eac9c765e66a8de8209 Signed-off-by: Ivan Vnučko <[email protected]>

Change-Id: Iad820292c7e6a8b881d8869e875b9643bdfcec61

Forgotten, not needed for s6-overlay solution Change-Id: Ib5f1c4708fb305a220185e8dc1cc97d6c8154486

OgarOgarovic · 2024-09-29T17:34:43Z

Created patchset upstream kolla-ansible/930841

#591 - WIP: Implement OVN IPSec

816e71b

fdobrovolny added the work in progress Pull requests that are work in progress, do not merge them label May 8, 2024

fdobrovolny marked this pull request as draft May 8, 2024 07:04

Alex-Welsh and others added 2 commits May 16, 2024 11:11

Automate prometheus blackbox configuration

91470d4

This change automates the prometheus blackbox monitoring configuration for common endpoints. Custom endpoints can be added to prometheus_blackbox_exporter_endpoints_custom. Change-Id: Id6f51a2bebee3ab63b84ca7032aad17c2933838c

#591 - WIP: Add OVS IPSec

eb52b84

matofeder reviewed Jul 3, 2024

View reviewed changes

fdobrovolny added 4 commits July 8, 2024 11:10

Fix end of lines

2e549a3

Rewrite event handler to xml

a584caf

This still does not work, however; for those coming after me, I'm pushing this, so you don't have to spend time on this version.

Changes the ipsec startup process

4316628

This should now avoid issues when restarting IPSEC

Enable XMLRPC in supervisor

77db963

OgarOgarovic mentioned this pull request Jul 10, 2024

Prepare PoC of Node to Node encryption using OVN in testbed SovereignCloudStack/issues#591

Open

9 tasks

OgarOgarovic self-assigned this Jul 18, 2024

artificial-intelligence and others added 5 commits September 6, 2024 15:14

Merge master into 591-ovn-ipsec

4161db2

openvswitch-vswitchd config.json fix

508850b

Change-Id: Ic28c945a1dda69813073eda8937b977ae97294ac

Clean up config.json template for openvswitch_ipsec service

0a2864a

Copied files are no longer needed for the s6-overlay based supervision and also command is set up empty as s6-overlay is managing processes in the container Change-Id: Iefe93965351b5adc8285042b3012a60ee4853140

Fix config command for kolla_start

b0c6e94

Change-Id: I6599761a68924b97bf62e32e66325f30b8b09c06

OgarOgarovic marked this pull request as ready for review September 24, 2024 07:29

OgarOgarovic changed the title ~~WIP: OVN IPSec~~ OVN IPSec Sep 24, 2024

OgarOgarovic removed the work in progress Pull requests that are work in progress, do not merge them label Sep 24, 2024

OgarOgarovic mentioned this pull request Sep 24, 2024

Add openvswitch-ipsec service image SovereignCloudStack/kolla#1

Open

matofeder approved these changes Sep 25, 2024

View reviewed changes

matofeder reviewed Sep 25, 2024

View reviewed changes

OgarOgarovic and others added 4 commits September 26, 2024 10:37

Merge branch 'master' into 591-ovn-ipsec

290e31a

Change-Id: I88da70757ae3a9bb2c3a506a064e4f1b7193b6c0

Remove remnants of the old supervisord solution

b530ab8

Removing scripts and their copy tasks which aren't used in s6-overlay solution. Change-Id: Ieec19d3f9095a23358ad8eac9c765e66a8de8209 Signed-off-by: Ivan Vnučko <[email protected]>

Merge "CI: increase timeout during server resize"

f8459c5

Zuul and others added 3 commits September 27, 2024 17:03

Merge "Automate prometheus blackbox configuration"

f35cf55

Merge branch 'master' into 591-ovn-ipsec

4318768

Change-Id: Iad820292c7e6a8b881d8869e875b9643bdfcec61

Remove supervisor event handler

72c6577

Forgotten, not needed for s6-overlay solution Change-Id: Ib5f1c4708fb305a220185e8dc1cc97d6c8154486

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OVN IPSec #4

OVN IPSec #4

fdobrovolny commented May 8, 2024 •

edited by matofeder

Loading

fdobrovolny commented Jun 5, 2024

matofeder left a comment •

edited

Loading

matofeder Sep 25, 2024

matofeder Sep 25, 2024

matofeder Sep 25, 2024

OgarOgarovic commented Sep 29, 2024

		@@ -0,0 +1,35 @@
		[supervisord]
		nodaemon=true

OVN IPSec #4

Are you sure you want to change the base?

OVN IPSec #4

Conversation

fdobrovolny commented May 8, 2024 • edited by matofeder Loading

fdobrovolny commented Jun 5, 2024

matofeder left a comment • edited Loading

Choose a reason for hiding this comment

matofeder Sep 25, 2024

Choose a reason for hiding this comment

matofeder Sep 25, 2024

Choose a reason for hiding this comment

matofeder Sep 25, 2024

Choose a reason for hiding this comment

OgarOgarovic commented Sep 29, 2024

fdobrovolny commented May 8, 2024 •

edited by matofeder

Loading

matofeder left a comment •

edited

Loading