So I found this 'quick start guide' https://gist.github.com/danielsnider/e09c56a96b187eec23c482f9400e59a1
which showed me where I needed to input my robot's IP address.
And now it finally moving!
Heres the guid for UR5 just incase the page goes all 404 on you!

SenseAct Quick Start
This guide has been tested on Ubuntu 16.04.

Install General Dependecies
sudo apt update
sudo apt install python3 python3-dev python3-pip python3-tk cmake libopenmpi-dev zlib1g-dev git
Install MuJuCo Robotics Simulator
The SenseAct learning task in simulation uses a popular physics simulator called MuJoCo. First install the prerequisites for MuJoCo:

sudo apt install patchelf libgl1-mesa-dev libgl1-mesa-glx libglew-dev libosmesa6-dev
Then install MuJoCo according to their install guide.

Sign up for a 30-day free trial on the MuJoCo website or free license if you are a student. The license key will arrive in an email with your username and password.
Download the MuJoCo version 1.50 binaries Linux, OSX, or Windows.
Unzip the downloaded mjpro150 directory into ~/.mujoco/mjpro150, and place your license key (the mjkey.txt file from your email) at ~/.mujoco/mjkey.txt.
Add MuJoCo to your PATH:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/dan/.mujoco/mjpro150/bin
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/dan/.mujoco/mjpro150/bin" >> ~/.bashrc
Install SenseAct
Download SenseAct:

git clone https://github.com/kindredresearch/SenseAct
cd SenseAct
(Optionally) Create a python virtual environment:

pip3 install virtualenv
virtualenv venv
source venv/bin/activate
Install SenseAct's python dependencies:

pip3 install -e .
pip3 install baselines==0.1.5
Run SenseAct
For your first learning task, we'll simulate the robotic control of a double inverted pendulum in a MuJuCo simulation.

Enable visualization of the simulated world in MuJoCo:

sed -i 's/is_render=False/is_render=True/g' ./examples/advanced/sim_double_pendulum.py
Start learning:

python3 ./examples/advanced/sim_double_pendulum.py
Hardware Setup
UR Robot Arm
Tested on UR Software v. 3.3.4.310. To control a UR robot arm you will need its IP address and be connected to the same network. You can find the IP address on the teach pendent under network settings.

Once you have the IP address of the arm enter it here:

SenseAct/examples/advanced/ur5_reacher.py

# Create UR5 Reacher2D environment
env = ReacherEnv(
        setup="UR5_default",
        host="192.168.5.20",              <-- Set IP here

Now start learning:

python3 examples/advanced/ur5_reacher.py
If you want to use a pretrained model:

git checkout load_model
python3 examples/advanced/ur5_reacher.py ./examples/advanced/pre_trained_models/ur_reacher_2_trpo.pkl

from senseact.