Cannot SSH into GitLab with VPN

I can’t establish a connection between my machine with a VPN and a private GitLab instance. I’ve setup the SSH key with:

ssh-keygen -t ed25519 -C "some comment"

And then copied the public key into a new SSH key in my GitLab settings.

When I try to establish a connection I see the below logs:

$ ssh -Tvvv git@<my_private_gitlab_host>
OpenSSH_9.6p1 Ubuntu-3ubuntu13.5, OpenSSL 3.0.13 30 Jan 2024
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/me/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/me/.ssh/known_hosts2'
debug2: resolving "<my_private_gitlab_host>" port 22
debug3: resolve_host: lookup <my_private_gitlab_host>:22
debug3: channel_clear_timeouts: clearing
debug3: ssh_connect_direct: entering
debug1: Connecting to <my_private_gitlab_host> [<ip address>] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x10

I’ve also tried to just telnet, results:

$ telnet <my_private_gitlab_host> 22
Trying <ip address>...
Connection failed: Connection timed out

I’m on Windows 11, WSL2 Ubuntu (though it also doesn’t work when trying to connect from outside the WSL VM)

Any ideas are welcome