I have recently compiled cosmos with DOMLIB=xcl. It works fine when I start a router configuration. But when I tried to stop the router instance with "cosmos stop click0", I got "Segmentation fault (core dumped)" from command line output. Does cosmos support stopping, suspending, and resuming a router instance right now?

I tried ponger.example. it worked.But after few seconds, I cannot get access to console(after "xl console clickos"), Then I found the cpu utilization of clickos is 100%. (maybe the reason )
I restart clickos. The cpu utilization of clickos soon reach 100%.
Xen version 4.4.1

Hi,

Recently, I am trying to make ClickOS run on Xen hypervisor with Ubuntu-14.04-64bits LTS desktop as Dom0. But I cannot work it out.

I have made it work on Debian-7.8-64bits. Seems it runs perfectly on Debian.
Just wondering whether the toolchain, cosmos or clickos is compatible with Ubuntu? Did you ever make it work on Ubuntu?

Here is a very similar problem : #3
I encoutered the same thing on Debian, but I made it work by following the instructions discussed in that issue. It is the "cosmos make DOMLIB=xl" that matters.
For Ubuntu, it did not work.

In Ubuntu, I do exactly the way I did on Debian:

1. Install Ubuntu 14.04 desktop as host.
2. Download Xen-4.4.1 source code, compile and install.
3. Reboot from Ubuntu-Xen-hypervisor with Ubuntu 14.04 as Dom0.
4. Git clickos, toolchain, cosmos, minios.
5. Make toolchain.
6. Make clickos
7. Make cosmos
8. Setup bridges for Xen.
9. xl create /home/work/clickos/minios/config.xen (This point, xl list shows clickos -b----. )
10. /home/work/cosmos/dist/bin/cosmos start $DOMID /home/work/clickos/minios/rule.click (This point, xl list show clickos -b---- .)
11. xl console $DOMID ($DOMID set correctly.)

After that the output is as following:

Xen Minimal OS!
start_info: 0x2ef000(VA)
nr_pages: 0xc00
shared_inf: 0x380fa000(MA)
pt_base: 0x2f2000(VA)
nr_pt_frames: 0x5
mfn_list: 0x2e9000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line:
stack: 0x241f40-0x261f40
MM: Init
_text: 0x0(VA)
_etext: 0xecd06(VA)
_erodata: 0x16a000(VA)
_edata: 0x16bb98(VA)
stack start: 0x241f40(VA)
_end: 0x2e8010(VA)
start_pfn: 2fa
max_pfn: c00
Mapping memory range 0x400000 - 0xc00000
setting 0x0-0x16a000 readonly
skipped 0x1000
MM: Initialise page allocator for 2fe000(2fe000)-c00000(c00000)
MM: done
Demand map pfns at c01000-2000c01000.
Heap resides at 2000c02000-4000c02000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0xc01000.
Initialising scheduler
Thread "Idle": pointer: 0x2000c02050, stack: 0x310000
Thread "xenstore": pointer: 0x2000c02800, stack: 0x320000
xenbus initialised on irq 1 mfn 0x294b8
Thread "shutdown": pointer: 0x2000c02fb0, stack: 0x330000
Dummy main: start_info=0x261f40
Thread "main": pointer: 0x2000c03760, stack: 0x340000
sparsing 0MB at 17e000
"main"

It is blocked here.
xl list still shows clickos with status -b----.

config.xen

kernel = './minios/build/clickos_x86_64'
vcpus = '1'

cpus = '2'

memory = '40'
vif = ['mac=00:15:17:15:5d:74,bridge=xenbr0']
name = 'click0'
on_crash = 'preserve'

rule.click

define ($MACARP 00:15:17:15:5d:74)
define ($IP 172.16.97.81)
in :: FromDevice();
out :: ToDevice();
c :: Classifier(
12/0806 20/0001, // ARP request
12/0800,
-);
arpr :: ARPResponder($IP $MACARP)
in -> c;
c[0] -> arpr -> out;
c[1] -> CheckIPHeader(14) -> Print("start") -> Discard();
c[2] -> Discard();

Many thanks!

--Hongda.

Currently Suspend/Resume is only supported for DOMLIB=xl

Hi,

I am trying to try the ponger example using your modified netback. I cannot seem to make it work. The console output for the ClickOS VM says:

Xen Minimal OS!
start_info: 0x1be000(VA)
nr_pages: 0x800
shared_inf: 0x8ce6b000(MA)
pt_base: 0x1c1000(VA)
nr_pt_frames: 0x5
mfn_list: 0x1ba000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line:
stack: 0x177040-0x197040
MM: Init
_text: 0x0(VA)
_etext: 0xe1e1d(VA)
_erodata: 0x15d000(VA)
_edata: 0x15ebb8(VA)
stack start: 0x177040(VA)
_end: 0x1b9010(VA)
start_pfn: 1c9
max_pfn: 800
Mapping memory range 0x400000 - 0x800000
setting 0x0-0x15d000 readonly
skipped 0x1000
MM: Initialise page allocator for 1cb000(1cb000)-800000(800000)
MM: done
Demand map pfns at 801000-2000801000.
Heap resides at 2000802000-4000802000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x801000.
Initialising scheduler
Thread "Idle": pointer: 0x2000802050, stack: 0x1e0000
Thread "xenstore": pointer: 0x2000802800, stack: 0x1f0000
xenbus initialised on irq 1 mfn 0x2222b0
Thread "shutdown": pointer: 0x2000802fb0, stack: 0x200000
Dummy main: start_info=0x197040
Thread "main": pointer: 0x2000803760, stack: 0x210000
sparsing 0MB at 176000
"main"
[on_status:205] router id 0
[on_status:206] status change to Running
Thread "click": pointer: 0x2000810c50, stack: 0x230000
backend dom 0
Mapping TX rings
0 map errors

Am I doing it right? The example.config file that I am using is as follows:

name = 'clickos'
kernel = '/home/nfv1/click/clickos/minios/build/clickos_x86_64'
vcpus = '1'

pinning your VCPU helps performance

cpus = '3'
memory = '8'

uncoment this line if you wish to use our backend

vif = ['bridge=vale0,script=vif-vale']

vif = ['mac=68:05:ca:29:2e:be,bridge=vale0']

on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'preserve'
click = 'ponger.click'

uncomment this line to enable PVH mode for ClickOS

pvh=1

Best wishes,
Hassan

Hi there.

I trying to make ponger test (https://cnp.neclab.eu/getting-started/), so I created a ponger.cfg file, but when I execute "cosmos create ponger.cfg" I have this error message:

[root@localhost ~]# cosmos create ponger.cfg
Parsing config file ponger.cfg
ponger.cfg:1: config parsing error near ($IP': lexical error ponger.cfg:2: config parsing error near($MAC': lexical error
ponger.cfg:4: config parsing error near ::': lexical error ponger.cfg:5: config parsing error near::': lexical error
ponger.cfg:6: config parsing error near //': lexical error ponger.cfg:7: config parsing error near::': lexical error
ponger.cfg:8: config parsing error near /0806': lexical error ponger.cfg:9: config parsing error near/0806': lexical error
ponger.cfg:10: config parsing error near /0800,': lexical error ponger.cfg:11: config parsing error near-);': lexical error
ponger.cfg:13: config parsing error near ::': lexical error ponger.cfg:14: config parsing error near::': lexical error
ponger.cfg:16: config parsing error near ->': lexical error ponger.cfg:17: config parsing error near->': lexical error
ponger.cfg:18: config parsing error near ->': lexical error ponger.cfg:19: config parsing error nearIdle': lexical error
ponger.cfg:20: config parsing error near ->': lexical error ponger.cfg:21: config parsing error near->': lexical error
ponger.cfg:22: config parsing error near `->': lexical error
warning: Config file looks like it contains Python code.
warning: Arbitrary Python is no longer supported.
warning: See http://wiki.xen.org/wiki/PythonInXlConfig
Failed to parse config: Invalid argument
[root@localhost ~]#

Well, following the ponger guide, the procedure is:

$ ifconfig xenbr0 10.10.0.1 netmask 255.255.255.0 up
$ cosmos create ponger.cfg
$ ping 10.10.0.3
# On another console you will the output corresponding to the elements
$ xl console ponger

But I think that the procedure should be:

cosmos create example.cfg (to create a ClickOS instance)
cosmos start clickos ponger.cfg (to put a Click Ponger configuration)

and so on.

I´m correct or do I should use "cosmos create ponger.cfg" command? If I can use "cosmos create ponger.cfg" command, how I put clickos instance configuration in this file?

Thank you.

libxcl was originally created for ClickOS guests, that do not require a FS to work.

Could be useful for other minimalistic guests to benefit from our stripped down toolstack. For that we need block device attach support.

After create and start a ClickOS VM with 'xl' and 'cosmos', I tried to open a ClickOS console with the command:

xl console clickos

However, the console stops and I can't access the VM. My terminal shows the following message:

Xen Minimal OS!
start_info: 0x1c0000(VA)
nr_pages: 0xc00
shared_inf: 0x2ffa4000(MA)
pt_base: 0x1c3000(VA)
nr_pt_frames: 0x5
mfn_list: 0x1ba000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line:
stack: 0x177040-0x197040
MM: Init
_text: 0x0(VA)
_etext: 0xe13fd(VA)
_erodata: 0x15d000(VA)
_edata: 0x15ebb8(VA)
stack start: 0x177040(VA)
_end: 0x1b9010(VA)
start_pfn: 1cb
max_pfn: c00
Mapping memory range 0x400000 - 0xc00000
setting 0x0-0x15d000 readonly
skipped 0x1000
MM: Initialise page allocator for 1cf000(1cf000)-c00000(c00000)
MM: done
Demand map pfns at c01000-2000c01000.
Heap resides at 2000c02000-4000c02000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0xc01000.
Initialising scheduler
Thread "Idle": pointer: 0x2000c02050, stack: 0x1e0000
Thread "xenstore": pointer: 0x2000c02800, stack: 0x1f0000
xenbus initialised on irq 1 mfn 0x124079
Thread "shutdown": pointer: 0x2000c02fb0, stack: 0x200000
Dummy main: start_info=0x197040
Thread "main": pointer: 0x2000c03760, stack: 0x210000
sparsing 0MB at 176000
"main"

And nothing else happens.
I'm don't know where is the problem, but it appears to be with cosmos.

Any idea about how to solve this problem?

root@nimbnode34:/home/wenhui/ClickOS/cosmos# sudo make
libxl: error: libxl.c:5098:libxl_get_physinfo: getting physinfo: Permission denied
libxl_physinfo failed.
libxl: error: libxl.c:5594:libxl_get_scheduler: getting domain info list: Permission denied

HELLO!
To start clickos on my NFV platform, I followed the steps showed on the wbepage ( http://cnp.neclab.eu/getting-started#clickos) and all chose the master branch from github, but something wrong happened.
I can't run clickos on ubuntu 14.04 and xen 4.4.4. I used "make DOMLIB=xcl" to build the cosmos. Then I input "cosmos create example.cfg" and "cosmos start clickos1 mirror.click" to create the clickos, but When I "xl console clickos", the information was showed as follows:

Xen Minimal OS!
start_info: 0x1d8000(VA)
nr_pages: 0x800
shared_inf: 0x103fe000(MA)
pt_base: 0x1db000(VA)
nr_pt_frames: 0x5
mfn_list: 0x1d4000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line:
stack: 0x1910c0-0x1b10c0
MM: Init
_text: 0x0(VA)
_etext: 0x100c46(VA)
_erodata: 0x17c000(VA)
_edata: 0x17db98(VA)
stack start: 0x1910c0(VA)
_end: 0x1d3010(VA)
start_pfn: 1e3
max_pfn: 800
Mapping memory range 0x1e3000 - 0x800000
setting 0x0-0x17c000 readonly
skipped 0x1000
MM: Initialise page allocator for 1e5000(1e5000)-800000(800000)
MM: done
Demand map pfns at 801000-2000801000.
Heap resides at 2000802000-4000802000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x801000.
Initialising scheduler
Thread "Idle": pointer: 0x2000802050, stack: 0x1f0000
Thread "xenstore": pointer: 0x2000802800, stack: 0x200000
xenbus initialised on irq 1 mfn 0xe0ab
Thread "shutdown": pointer: 0x2000802fb0, stack: 0x210000
Dummy main: start_info=0x1b10c0
Thread "main": pointer: 0x2000803760, stack: 0x220000
sparsing 0MB at 190000
"main"
[on_status:217] router id 0
[on_status:218] status change to Running
Thread "click": pointer: 0x2000810a00, stack: 0x240000
Failed to read /local/domain/0/backend/vif/2/0/feature-netmap.
************************ NETFRONT for device/vif/0 **********

net TX ring size 256, 1024 KB
net RX ring size 256, 1024 KB buffer space
split event channels enabled
backend at /local/domain/0/backend/vif/2/0
mac is 00:15:17:15:5d:74
[on_status:217] router id 1
[on_status:218] status change to Running
Thread "click": pointer: 0x200081c840, stack: 0x460000
Failed to read /local/domain/0/backend/vif/2/0/feature-netmap.
************************ NETFRONT for device/vif/0 **********

net TX ring size 256, 1024 KB
net RX ring size 256, 1024 KB buffer space
split event channels enabled
backend at /local/domain/0/backend/vif/2/0
mac is 00:15:17:15:5d:74

Different with the output on the webpage(http://cnp.neclab.eu/getting-started#clickos) when input "xl console clickos", there is no driver which is running on my computer.

My example.cfg:
name = 'clickos1'
kernel = '/home/wr/clickos_x86_64'
vcpus = '1'

pinning your VCPU helps performance

#cpus = '3'
memory = '8'

uncoment this line if you wish to use our backend

#vif = ['bridge=vale0,script=vif-vale']
vif = ['mac=00:15:17:15:5d:74,bridge=xenbr0']

on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'preserve'
click = 'mirror.click'

uncomment this line to enable PVH mode for ClickOS

#pvh=1
~

My mirroe.click
FromDevice -> EtherMirror -> ToDevice

Please help! THANKS!

Though not necessary but it can make things easier for xenstore if we properly release a domain, when shutting down from the toolstack (i.e. not initiated by the guest or crash).

Hey guys,

Even correcting the order of linker options with XEN 4.4.1 (which worked for cosmos) did not help me in compiling the cosmos python API.

I tried compiling the code manually... no success!

gcc -fPIC -c clickos.c cosmos_wrap.c -I/usr/include/python2.7 -I/home/cosmos/src -I/home/cosmos/include -I/home/cosmos/include/xcl -I/home/xen-4.4.1/tools/libxl -I/home/xen-4.4.1/tools/libxc -I/home/xen-4.4.1/tools/include -I/home/xen-4.4.1/tools/xenstore
or only ../../src/main.c
swig -includeall -I/home/cosmos/include -python cosmos.i
ld -shared clickos.o cosmos_wrap.o -o _cosmos.so

Do you have any suggestion. OR would you mind telling me the version of XEN, Linux kernel and click-os that you used?

Thanks,

XCL doesn't properly parse domain IDs from the command line, only names.

Trying to get clickOS up and running, and hitting a brick wall when running the makefile for cosmos. I'm quite new to xen in general so I could just be hitting some really naive error and be unaware of it; if that's the case then please excuse my inexperience.

Here's the output of the makefile:

jeff@JeffSDNLaptop:~/cosmos$ sudo make
make: *** xen: No such file or directory.  Stop.
  LD  /home/jeff/cosmos/dist/bin/cosmos
/home/jeff/cosmos/build/libcosmos/clickos.o: In function `xenstore_init':
clickos.c:(.text+0xc): undefined reference to `xs_daemon_open'
clickos.c:(.text+0x49): undefined reference to `xs_get_domain_path'
/home/jeff/cosmos/build/libcosmos/clickos.o: In function `clickos_read_handler':
clickos.c:(.text+0x135): undefined reference to `xs_transaction_start'
clickos.c:(.text+0x161): undefined reference to `xs_write'
clickos.c:(.text+0x17d): undefined reference to `xs_transaction_end'
clickos.c:(.text+0x19a): undefined reference to `xs_watch'
clickos.c:(.text+0x1cb): undefined reference to `xs_fileno'
clickos.c:(.text+0x204): undefined reference to `xs_transaction_start'
clickos.c:(.text+0x226): undefined reference to `xs_read'
clickos.c:(.text+0x246): undefined reference to `xs_transaction_end'
clickos.c:(.text+0x28d): undefined reference to `xs_unwatch'
clickos.c:(.text+0x2a2): undefined reference to `xs_transaction_start'
clickos.c:(.text+0x2ce): undefined reference to `xs_write'
clickos.c:(.text+0x2ea): undefined reference to `xs_transaction_end'
/home/jeff/cosmos/build/libcosmos/clickos.o: In function `clickos_write_handler':
clickos.c:(.text+0x3a3): undefined reference to `xs_write'
/home/jeff/cosmos/build/libcosmos/clickos.o: In function `clickos_start':
clickos.c:(.text+0x5c8): undefined reference to `xs_transaction_start'
clickos.c:(.text+0x636): undefined reference to `xs_read'
clickos.c:(.text+0x709): undefined reference to `xs_write'
clickos.c:(.text+0x732): undefined reference to `xs_write'
clickos.c:(.text+0x75b): undefined reference to `xs_set_permissions'
clickos.c:(.text+0x784): undefined reference to `xs_set_permissions'
clickos.c:(.text+0x7bb): undefined reference to `xs_write'
clickos.c:(.text+0x90c): undefined reference to `xs_write'
clickos.c:(.text+0x948): undefined reference to `xs_transaction_end'
clickos.c:(.text+0x96f): undefined reference to `xs_transaction_start'
clickos.c:(.text+0x99e): undefined reference to `xs_write'
clickos.c:(.text+0x9ba): undefined reference to `xs_transaction_end'
/home/jeff/cosmos/build/libcosmos/clickos.o: In function `clickos_stop':
clickos.c:(.text+0xa4b): undefined reference to `xs_transaction_start'
clickos.c:(.text+0xa77): undefined reference to `xs_write'
clickos.c:(.text+0xa93): undefined reference to `xs_transaction_end'
collect2: error: ld returned 1 exit status
make: *** [/home/jeff/cosmos/dist/bin/cosmos] Error 1

For reference, I've followed the instructions as found here: http://cnp.neclab.eu/getting-started#clickos

I've successfully built the image, and am currently at the step of

For the toolstack, just clone the toolstack and run make.

Is this error the result of something like having mismatching versions of xen installed? xl info tells me I'm running 4.4.1, and I've downloaded the sources for 4.4.1.

Any help/advice would be appreciated!

Every command is made on the context of a domain name with any of the DOMLIB built in.

With DOMLIB=none we use domain id for this, though it causes some confusion when using cosmos.

We can fix this by adding a very simple implementation to domain_none based on xcl.

when I use xl list:
Name ID Mem VCPUs State Time(s)
Domain-0 0 5876 4 r----- 235.9
clickos 1 8 1 -b---- 0.0

then I use cosmos start $DOMID mirror.click:
Domain ID for 1: -22
Location of click script: mirror.click

but when I input xl console clickos，however it blocked
Xen Minimal OS!
start_info: 0x2e6000(VA)
nr_pages: 0x800
shared_inf: 0x845cf000(MA)
pt_base: 0x2e9000(VA)
nr_pt_frames: 0x5
mfn_list: 0x2e2000(VA)
mod_start: 0x0(VA)
mod_len: 0
flags: 0x0
cmd_line:
stack: 0x23af40-0x25af40
MM: Init
_text: 0x0(VA)
_etext: 0xe8826(VA)
_erodata: 0x163000(VA)
_edata: 0x164bb8(VA)
stack start: 0x23af40(VA)
_end: 0x2e1010(VA)
start_pfn: 2f1
max_pfn: 800
Mapping memory range 0x400000 - 0x800000
setting 0x0-0x163000 readonly
skipped 0x1000
MM: Initialise page allocator for 2f3000(2f3000)-800000(800000)
MM: done
Demand map pfns at 801000-2000801000.
Heap resides at 2000802000-4000802000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x801000.
Initialising scheduler
Thread "Idle": pointer: 0x2000802050, stack: 0x300000
Thread "xenstore": pointer: 0x2000802800, stack: 0x310000
xenbus initialised on irq 1 mfn 0x1ba64c
Thread "shutdown": pointer: 0x2000802fb0, stack: 0x320000
Dummy main: start_info=0x25af40
Thread "main": pointer: 0x2000803760, stack: 0x330000
sparsing 0MB at 177000
"main"

could anyone help me? I have tried it for one day.

I am trying to load a large generated click configuration using cosmos start $DOMID script.click, and i get an error saying that my file is too large.

Click configurations with more than 15000 bytes not supported.

In the code that the limit is hard coded:

Why is there a file size limit ? Is there another way to load a large script ?

Hello,

I'm trying to compile cosmos, problem I'm current facing is xen header file is not found.

jinxuanw@jinxuanw-Aspire-4820TG:~/click/cosmos$ sudo make
make: *** xen: No such file or directory.  Stop.
  CC  /home/jinxuanw/click/cosmos/src/clickos.c
/home/jinxuanw/click/cosmos/src/clickos.c:48:22: fatal error: xenstore.h: No such file or directory
 #include <xenstore.h>
                      ^
compilation terminated.
Makefile:240: recipe for target '/home/jinxuanw/click/cosmos/build/libcosmos/clickos.o' failed
make: *** [/home/jinxuanw/click/cosmos/build/libcosmos/clickos.o] Error 1

I do have xen source code configured

~/click/xen-4.4.0$ sudo ./configure 
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
Will build the following subsystems:
  xen
  kernels
  tools
  stubdom
  docs
configure: creating ./config.status
config.status: creating ./config/Toplevel.mk
=== configuring in tools (/home/jinxuanw/click/xen-4.4.0/tools)
configure: running /bin/bash ./configure --disable-option-checking '--prefix=/usr/local'  --cache-file=/dev/null --srcdir=.
**Omitted**
configure: creating ./config.status
config.status: creating ../config/Docs.mk

I also have make install in the tools library.

jinxuanw@jinxuanw-Aspire-4820TG:~/click/xen-4.4.0/tools$ sudo make -C include/
make: Entering directory '/home/jinxuanw/click/xen-4.4.0/tools/include'
make -C xen-foreign
make[1]: Entering directory '/home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign'
python mkheader.py arm32 arm32.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/arch-arm.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/xen.h
python mkheader.py arm64 arm64.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/arch-arm.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/xen.h
python mkheader.py x86_32 x86_32.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/arch-x86/xen-x86_32.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/arch-x86/xen.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/xen.h
python mkheader.py x86_64 x86_64.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/arch-x86/xen-x86_64.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/arch-x86/xen.h /home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign/../../../xen/include/public/xen.h
python mkchecker.py checker.c arm32 arm64 x86_32 x86_64
gcc -Wall -Werror -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -Wdeclaration-after-statement -o checker checker.c
./checker > tmp.size
diff -u reference.size tmp.size
rm tmp.size
make[1]: Leaving directory '/home/jinxuanw/click/xen-4.4.0/tools/include/xen-foreign'
mkdir -p xen/libelf
ln -sf /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/COPYING xen
ln -sf /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/vcpu.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/sysctl.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/kexec.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/dom0_ops.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/xen.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/gcov.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/arch-arm.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/nmi.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/xencomm.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/xenoprof.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/event_channel.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/version.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/mem_event.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/memory.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/arch-x86_32.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/features.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/sched.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/callback.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/grant_table.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/xen-compat.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/physdev.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/arch-x86_64.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/platform.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/tmem.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/elfnote.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/trace.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/domctl.h xen
ln -sf /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/arch-x86 /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/hvm /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/io /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/public/xsm xen
ln -sf ../xen-sys/Linux xen/sys
ln -sf /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/xen/libelf.h /home/jinxuanw/click/xen-4.4.0/tools/include/../../xen/include/xen/elfstructs.h xen/libelf/
ln -s ../xen-foreign xen/foreign
touch xen/.dir
make: Leaving directory '/home/jinxuanw/click/xen-4.4.0/tools/include'

$XEN_ROOT is also setup correctly.

jinxuanw@jinxuanw-Aspire-4820TG:~/click/xen-4.4.0/tools$ echo $XEN_ROOT
/home/jinxuanw/click/xen-4.4.0

Also based on xl info I see my Xen version installed is 4.4.0. I think I download the correct version of Xen source code.

I think I have follow all steps provided in the README.md. But I still couldn't solve this compile error. Hope I could get some suggestions that might help me successfully compile cosmos.

Thank you.

To avoid having link errors with recent toolchains mentioned on #12 it is required to make sure of the correct order of linker options in the Makefile.
It is a known issue on Xen with regards to its tools particularly with the xenstore:
http://lists.xen.org/archives/html/xen-devel/2010-07/msg01538.html

I was able to fix the issue by changing the Makefile like that:

#########################################################################\

Configure building

#########################################################################\

CDEFINES += -DXEN_VERSION=$(XEN_VERSION)

CINCLUDES += -I$(INCLUDE_DIR)

CFLAGS += -std=gnu99
CFLAGS += -fno-strict-aliasing
CFLAGS += -fPIC
CFLAGS += -Wall -Wstrict-prototypes
CFLAGS += -Wno-format-zero-length

LDFLAGS += -lxenctrl-----------------------------remove this line

LDFLAGS += -lxenguest-----------------------------remove this line

LDFLAGS += -lxenstore-----------------------------remove this line

LDFLAGS += -luuid -ldl -lutil-----------------------------remove this line

COS_LDFLAGS += -ldl -lutil-----------------------------add this line
COS_LDFLAGS += -lxenctrl-----------------------------add this line
COS_LDFLAGS += -lxenguest-----------------------------add this line
COS_LDFLAGS += -lxenstore-----------------------------add this line
COS_LDFLAGS += -luuid-----------------------------add this line

and then change the way the linker is called in the sections that I want, including:

#########################################################################\

"libxcl" section, "libcosmos" section and "cosmos" section

#########################################################################\

.
.

$(call cclink,$^ ,'LD ')-----------------------------remove this line

$(call cclink,$^ $(COS_LDFLAGS),'LD ')-----------------------------add this line
.
.

Currently my problem is creating the python wrapper that works. But it still doesn't.

Running xl info in order to define XEN_HOST_VERSION might require sudo.

after run 'make' in cosmos,
I tried "cosmos start clickos.click."
but i got " -bash: cosmos: command not found "

Please help me

Currently there is no PVH with either of the DOMLIB.

Booting with PVH means:

• Enabling HAP;
• Using native TSC mode;
• Properly setting PVH features;
• Not doing the phys to machine translation since it's an autotranslated guest;