nakato / nixos-bpir3-example Goto Github PK

View Code? Open in Web Editor NEW

20.0 20.0 5.0 69 KB

License: MIT License

Nix 100.00%

nixos-bpir3-example's People

Contributors

Stargazers

Watchers

Forkers

ghostbuster91 keegandahm albertov steveej-forks aredridel

nixos-bpir3-example's Issues

Switching to more recent kernel

Hi,

This repository currently uses fixed version of linux kernel - "6.4.0-rc5".
As "6.4.0" has been recently release I would like to switch to it.

Now, as you explained in another issue, we cannot just switch to the upstream kernel as we need to patch few things:

dto
nvme patch
kernel config

dto

Based on the 6.4 tag it seems that some of them are no longer needed:

mt7986a-bananapi-bpi-r3-nand.dts
mt7986a-bananapi-bpi-r3-sd.dts

Am I right?

nvme patch

This is still required as the affected file hasn't been changed for quite some time. No action needed.

kernel config

This is the one I am the most puzzled about. This current config was generated for 6.3.0-rc2 so it does not even match the version of linux kernel that is in use. I guess that is not a big issue as this config is likely compatible between minor versions. Is it enough to leave it as it is, or should we regenerate new one for 6.4? If so how do one go about that and what options should be changed from the default one?

consider contribution to nixos-aarch64 image

Hi, thanks for that repo, this is really awesome, and thanks for the prebuild image!

I was wondering if it would be possible to upstream that build into https://github.com/Mic92/nixos-aarch64-images though I must admit that my knowledge about this stuff is very limited, so it may be as well not a good idea.

Question regarding cross-compilation

I read that cross-compilation with nix is possible (though it might not always work).
I wanted to try it in this repository but whatever I do I am always getting an error that my nix-daemon cannot build aarch64 which means that I failed do configure cross-compilation.

I wonder if you tried it?

I also came across your post in bananapi forum where you said:

Nix doesn’t do cross-compilation, so any aarch64 system with enough memory for the nix command to and still have enough memory for the compiler to operate. 1GB might work, but you probably are better off with 2GB or more.

Which confuses me quite a bit. Could you shed some light on that?

No space left on device while building SD card image

Hi, I flashed your image on the SD card and booted BPI from it.

Then I wanted to build my own image using your flake and recent changes. I cloned the repository and run the command from the readme.

Unfortunately it failed with a following error:

linux> fixdep: not all data was written to the output
linux> make[2]: *** [/nix/store/nmkqiyicgv8636g6fkv1gpw3kzsxq8nm-linux-6.4.0-rc5-dev/lib/modules/6.4.0-rc5/source/scripts/Makefile.host:127: scripts/kconfig/util.o] Error 1
linux> make[2]: *** Deleting file 'scripts/kconfig/util.o'
linux> make[1]: *** [/nix/store/nmkqiyicgv8636g6fkv1gpw3kzsxq8nm-linux-6.4.0-rc5-dev/lib/modules/6.4.0-rc5/source/Makefile:692: oldconfig] Error 2
linux> make: *** [Makefile:226: __sub-make] Error 2
uboot-mt7986a_bpir3_sd_defconfig> FATAL ERROR: Error writing device tree blob: No space left on device
uboot-mt7986a_bpir3_sd_defconfig> Check /build/u-boot-2023.07-rc3/arch/arm/dts/.mt7623a-unielec-u7623-02-emmc.dtb.pre.tmp for errors
uboot-mt7986a_bpir3_sd_defconfig> make[2]: *** [scripts/Makefile.lib:335: arch/arm/dts/mt7623a-unielec-u7623-02-emmc.dtb] Error 1
uboot-mt7986a_bpir3_sd_defconfig> make[2]: *** Deleting file 'arch/arm/dts/mt7623a-unielec-u7623-02-emmc.dtb'
uboot-mt7986a_bpir3_sd_defconfig> make[2]: *** Waiting for unfinished jobs....
error: builder for '/nix/store/8wfzrm14m6iphdc9mm27mmjban89v9rf-linux-6.4.0-rc5.drv' failed with exit code 2
       note: build failure may have been caused by lack of free disk space
error: 1 dependencies of derivation '/nix/store/cl7p91bhs4z4p5lkmxn5rh58y0dbqand-nixos-system-bpir3-23.05.20230506.897876e.drv' failed to build
error: 1 dependencies of derivation '/nix/store/y3m2hs8iybr1kj7dplafx05h5vksp432-nixos-bananapir3-sd.drv' failed to build

Any idea why that happened? My SD card is 120GB and I don't have nvme disk.
Here is an output from df:

 df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        100M     0  100M   0% /dev
tmpfs           995M     0  995M   0% /dev/shm
tmpfs           498M  3.0M  495M   1% /run
tmpfs           995M  384K  995M   1% /run/wrappers
/dev/mmcblk0p6  117G  5.3G  107G   5% /
/dev/mmcblk0p5  511M   31M  481M   7% /boot
tmpfs           199M   12K  199M   1% /run/user/0

Just letting you (and other people) know the kernel config has changed in 6.6

CONFIG_MT7986_WMAC -> CONFIG_MT798X_WMAC

See:

what driver is responsible for mmcblk devices?

I'm trying to boot NixOS on BPI-R3, here's my config and packages. It mostly works, but doesn't boot because /dev/mmcblk0 doesn't appear. I assume that's because I'm using the default kernel (linux_testing), I saw someone else using it so I figured it'd be fine.

Do you have any ideas why the device isn't appearing? I made sure mmc_block kernel module is loaded. Comparing dmesg logs, my logs lack mtk-pcie output, but I'm not sure what to do with this info. Do I have to compile a custom kernel, or can I use the default one?

enable hardware flow offloading

Hi,

I was trying to get the hardware flow offloading using nftables to work but I was getting some error.

  > ruleset.conf:5:15-15: Error: Could not process rule: No such file or directory
       >     flowtable f {
       >               ^
       > ruleset.conf:13:28-42: Error: Could not process rule: No such file or directory
       >   ip protocol { tcp, udp } flow offload @f
       >                            ^^^^^^^^^^^^^^^

Frank, from bananapi forum, suggested that it might be that CONFIG_NF_FLOW_TABLE is not enabled.
After looking at the config file I see that it is enabled though CONFIG_NF_FLOW_TABLE_INET is commented out.

Was there any reason to disable it? Can I simply uncomment it and assume that flow offloading should start to work?

No space left on a device

Hi, today I noticed that my router became very sluggish.

After a quick inspection it turned out that during the night it tried to run nixos upgrade but didn't have enough space on the /boot partition.

Aug 17 04:17:07 surfer nixos-upgrade-start[2995908]: cat: write error: No space left on device
Aug 17 04:17:07 surfer nixos-upgrade-start[2993981]: warning: error(s) occurred while switching to the new configuration
Aug 17 04:17:07 surfer systemd[1]: nixos-upgrade.service: Main process exited, code=exited, status=1/FAILURE
Aug 17 04:17:07 surfer systemd[1]: nixos-upgrade.service: Failed with result 'exit-code'.
Aug 17 04:17:07 surfer systemd[1]: Failed to start NixOS Upgrade.
Aug 17 04:17:07 surfer systemd[1]: nixos-upgrade.service: Consumed 5min 34.070s CPU time, received 39.6M IP traffic, sent 338.6K IP traffic.
q

$ df -h
/dev/mmcblk0p6   59G   26G   30G  47% /
/dev/nvme0n1p3   37G  440K   35G   1% /var
/dev/nvme0n1p1   19G  286M   17G   2% /var/log
/dev/mmcblk0p5  511M  511M     0 100% /boot
/dev/nvme0n1p2   37G   84K   35G   1% /tmp
tmpfs           196M     0  196M   0% /run/user/1000

What can be done in such a case? A quick search reveals that this is was a known issue for RPI NixOS/nixpkgs#23926 (comment) and it was solved by moving the boot partition onto the main rootfs but this is not possible in our case.
What should we do to prevent this from happening? I did have an automatic gc set to work weekly https://github.com/ghostbuster91/nixos-router/blob/main/nixos/configuration.nix#L66

I tried running manually garbage collection and it removed some old generations but the boot partition was still full.

Cannot get wifi to work

Hi,

Have you managed to get the wifi working?

I tried first with wpa_supplicant, but the iwconfig was showing:

> iwconfig                                                                                                            
lo        no wireless extensions.
eth0      no wireless extensions.
eth1     no wireless extensions.
wan       no wireless extensions.
lan0      no wireless extensions.
lan1      no wireless extensions.
lan2      no wireless extensions.
lan3      no wireless extensions.
lan4      no wireless extensions.
wlan0     no wireless extensions.
wlan1     no wireless extensions.
br0       no wireless extensions.

Here is a relevant section of my configuration for supplicant:

   wireless = {
      enable = true;
      userControlled.group = "network";
      interfaces = [ "wlan0" "wlan1" ];
      networks = {
        echelon = {
         # SSID with no spaces or special characters
         psk = "abcdefgh"; # (password will be written to /nix/store!)
        };
     
      };
     };

Then I learned that for bridging wlan interfaces with eth I will need to use hostapd anyway.
I tried to get it running using following configuration:

 services.hostapd = {
   enable = true;
   interface = "wlan0"; # 2.4
   ssid="bpir3wifi";
   hwMode="g";
   driver="nl80211";
   group="network";
   countryCode="PL";
   logLevel=0;
 };

but it always fail with not very verbose error:

Jun 16 00:18:17 bpir3 systemd[1]: Started hostapd wireless AP.
Jun 16 00:18:17 bpir3 hostapd[7227]: 1 errors found in configuration file '/nix/store/9bhygzp6ma5sa63bh0fbk9npj0mph92n-hostapd.>
Jun 16 00:18:17 bpir3 hostapd[7227]: Failed to set up interface with /nix/store/9bhygzp6ma5sa63bh0fbk9npj0mph92n-hostapd.conf
Jun 16 00:18:17 bpir3 hostapd[7227]: Failed to initialize interface
Jun 16 00:18:17 bpir3 systemd[1]: hostapd.service: Main process exited, code=exited, status=1/FAILURE
Jun 16 00:18:17 bpir3 systemd[1]: hostapd.service: Failed with result 'exit-code'.
Jun 16 00:18:17 bpir3 systemd[1]: hostapd.service: Scheduled restart job, restart counter is at 5.
Jun 16 00:18:17 bpir3 systemd[1]: Stopped hostapd wireless AP.
Jun 16 00:18:17 bpir3 systemd[1]: hostapd.service: Start request repeated too quickly.
Jun 16 00:18:17 bpir3 systemd[1]: hostapd.service: Failed with result 'exit-code'.
Jun 16 00:18:17 bpir3 systemd[1]: Failed to start hostapd wireless AP.

The generated hostapd file looks as follows:

interface=wlan0
driver=nl80211
ssid=bpir3wifi
hw_mode=g
channel=7
ieee80211n=1
ieee80211ac=1
country_code=PL
ieee80211d=1

# logging (debug level)
logger_syslog=-1
logger_syslog_level=0
logger_stdout=-1
logger_stdout_level=0

ctrl_interface=/run/hostapd
ctrl_interface_group=network

wpa=2
wpa_pairwise=CCMP
wpa_passphrase=my_sekret

I also tried using different drivers - mt76 but it always claims that

: Line 2: invalid/unknown driver 'mt76'

lshw reports that the driver is: driver=mt7986-wmac but switching to it doesn't change anything

lsmod reports that module mt76 was loaded correctly:

❯ lsmod                                                                                                               00:32:34
Module                  Size  Used by
8021q                  28672  0
garp                   16384  1 8021q
mrp                    20480  1 8021q
crct10dif_ce           12288  1
xt_LOG                 12288  2
nf_log_syslog          16384  2
xt_tcpudp              12288  0
nft_compat             16384  2
sch_fq_codel           16384  17
nf_tables             225280  21 nft_compat
libcrc32c              12288  1 nf_tables
nfnetlink              20480  2 nft_compat,nf_tables
tun                    57344  0
tap                    28672  0
macvlan                28672  0
fuse                  135168  1
ip_tables              32768  0
x_tables               40960  4 nft_compat,xt_LOG,xt_tcpudp,ip_tables
mt7915e               155648  0
mt76_connac_lib        65536  1 mt7915e
mt76                   86016  2 mt7915e,mt76_connac_lib
mac80211              557056  3 mt76,mt7915e,mt76_connac_lib
libarc4                12288  1 mac80211
cfg80211              389120  4 mt76,mt7915e,mac80211,mt76_connac_lib

Last but not least I found this comment:
https://github.com/nakato/nixos-bpir3-example/blob/main/bpir3-dts/mt7986a-bananapi-bpi-r3-wirless.dts#L11

So is there anything that we can do about it?

error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)

Hi,

Whenever I try to use nix-index or nix-shell and such I am getting following error:

❯ nix-index                                                                                                           22:57:14
+ querying available packages
error: querying available packages failed
caused by: nix-env failed with error: nix-env failed with exit code 1:
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels/nixos' does not exist, ignoring
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at «none»:0: (source not available)

Which is presumably because the NIX_PATH environment variable is not set.
I guess that the reason why it is not set is that the nixos was never "installed" but rather we are running an "installation disk"(?).

Do you know how that could be fixed?

My desktop nixos reports:

> echo $NIX_PATH
nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels

However these paths do not exist on the bananpi nixos.