ironicbadger / pms-wiki Goto Github PK
View Code? Open in Web Editor NEWThe aim is to share knowledge and information about building an open-source media server.
Home Page: https://perfectmediaserver.com
License: Other
The aim is to share knowledge and information about building an open-source media server.
Home Page: https://perfectmediaserver.com
License: Other
Hello!
I hate being "that guy" and I know Soon is just whenever you get to it, and you probably have a heap of other stuff on the go - but any guideline on when (or if) you'll have a chance to pull together the Proxmox manual install guide? I'm happy to be a beta tester if that would be helpful...
If it's not going to be a thing or it's going to be a while off, I'll just bite the bullet and wing it, but I do like having a good example to refer to!
Presumably it's not vastly different to the Ubuntu build (other than some stuff being prebuilt due to the nature of Proxmox etc) so I could use that as a exemplar?
The old nextcloud.com/try
link doesnt work anymore. The new correct one seems to be try.nextcloud.com
,
The go to repo
header item when clicked -> presently opens in the current window/tab.
Recommend making this link open in a NEW window/tab, if possible \/
Lines 2 to 4 in 356953e
Hello!
First off, thanks for a great site and resources for setting up a perfect media server!
I'm about to setup my own PMS following your guides here. I went and installed Proxmox and spun up a LXC container with Ubuntu and now trying to figure out how I pass in the drives and setup mergerfs in the best way but noticed that I can't pass in full disk as is into a LXC or maybe I'm missing something, but seems I can only bind directories? How have you set it up?
Are there something to be extra careful with here to not bit yourself in the foot later, something that doesn't play nice with mergerfs or so?
Thanks again!
I've forked the PMS wiki to add my own review of a couple chassis I've worked with over the years, and saw the CS380 already posted there - I've had some extensive experience with this case, modding it due to some design issues with air flow, along with some backplane/sled concerns, but wasn't sure how one might go about adding such details to an existing review.
How (if at all that is) would you prefer such things be handled?
Hi! I love PMS and frequently come back to reread it. I love how it continues to change and grow. Something that I'm learning about even more with this whole fun adventure is server security. As you showed, using a Bastion and other security features is recommended for any public facing server. Can you point users like me to resources for how to set up security for this kind of machine more clearly? The https://github.com/geerlingguy/ansible-role-security repo seems like it is a good start, but not complete. I read everywhere that you should have security, but rarely do I find detailed instructions on how to actually set it up and what to run.
I've learned a little about Snort (and Pulled Pork to accompany it). Are they necessary? Recommended? How to implement them correctly/easily? What about Firewalls? As a layman (and I consider myself pretty smart and learned on stuff like this), I still struggle with how to set up a firewall correctly, especially at home without spending hundreds of dollars on expensive equipment? Does the bastion and firewall have to be physically different servers? Or could it be a Proxmox virtual machine on the main server?
When setting up i had issues getting write permission working on a windows PC that was connected. I could read the directory but not write to it despite having read only = no
and/or writeable =yes
. As i understand this is quite a common issue but the guide as is would indicate it should work at that point.
I ended up fixing it by giving myself ownership of the samba directory and adding in a create mask and directory mask to the samba config.
I can provide more details and/or submit a PR but i'm not sure if what i did was best practice, i was an edge case and this isn't common, or if there's a better way to fix this issue.
On https://perfectmediaserver.com/tech-stack/snapraid/ the image for MergerFS SnapRAID diagram is broken.
It points to https://perfectmediaserver.com/tech-stack/images/diagram-mergerfs-snapraid.png where it should point to https://perfectmediaserver.com/images/diagram-mergerfs-snapraid.png.
It appears this change broke it: 6d7f3fe
On git hub (https://github.com/ironicbadger/pms-wiki/blob/main/docs/tech-stack/snapraid.md) ../images works
but on the pms site (https://perfectmediaserver.com/tech-stack/snapraid/) it needs to be ../../images
Just a suggestion to add 'category.create=mfs' to the mergerfs mount command in /etc/fstab. Otherwise it defaults to epmfs, which is not optimal if you are starting from empty drives - which this tutorial assumes.
https://github.com/trapexit/mergerfs#what-policies-should-i-use:
Why are all my files ending up on 1 drive?!
Did you start with empty drives? Did you explicitly configure a category.create policy? Are you using an existing path / path preserving policy?The default create policy is epmfs. That is a path preserving algorithm. With such a policy for mkdir and create with a set of empty drives it will select only 1 drive when the first directory is created. Anything, files or directories, created in that first directory will be placed on the same branch because it is preserving paths.
This catches a lot of new users off guard but changing the default would break the setup for many existing users. If you do not care about path preservation and wish your files to be spread across all your drives change to mfs or similar policy as described above. If you do want path preservation you'll need to perform the manual act of creating paths on the drives you want the data to land on before transferring your data. Setting func.mkdir=epall can simplify managing path preservation for create. Or use func.mkdir=rand if you're interested in just grouping together directory content by drive.
Recommend setting cache.files from off to cache.files=partial as =off will not play nicely with some services, such as torrents. from the trapexit/merferfs wiki:
BASIC SETUP
If you don't already know that you have a special use case then just start with one of the following option sets.
You need mmap (used by rtorrent and many sqlite3 base software)
allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mf
Example where cache.files=off causing issues:
linuxserver/docker-deluge#140 (comment)
=off actually worked fine for me with podman until I switched back to docker where I ran into this issue, though it's possible it was a recent update that caused this as I just pulled new images due to the switch
Hi, it seems that the picture describing the setup of SnapRAID
with mergerfs
is missing. Can you add it back? The pictures really help to visualise a lot better. Thanks :)
Link to file: https://github.com/ironicbadger/pms-wiki/blob/main/docs/tech-stack/snapraid.md
Having the https://github.com/n1trux/awesome-sysadmin referenced as well would be a great addition to the top10apps page
I think it is useful to document how the storage pool would be accessed inside the VM’s. Is this done through Samba or NFS in your current setup?
This is pathetic
Hi,
Do you still use mergerfs? What fstab entry do you have today and why?
PMS wiki has this "example":
/mnt/disk* /mnt/storage fuse.mergerfs defaults,nonempty,allow_other,use_ino,cache.files=off,moveonenospc=true,category.create=mfs,dropcacheonclose=true,minfreespace=250G,fsname=mergerfs 0 0
However, this seems to slightly differ from the "recommended defaults" mentioned in the mergerfs repo:
/mnt/hdd0:/mnt/hdd1 /media mergerfs cache.files=partial,dropcacheonclose=true,category.create=mfs 0 0
Could you elaborate on your current setup, if you're still using mergerfs. And if not - what did you switch to? Zfs?
Thanks :)
The site footer reads:
Licensed under GPLv3
The question becomes:
Is this referring to the content? The software?
If the license is intended for the content, then... isn't GPL a software-specific license, and wouldn't something like Creative Commons be more appropriate?
I've been looking at replacing unraid and came across your wiki (very helpful btw). It has given me several ideas to roll-my-own ubuntu box to replace unraid with snapraid. There is one piece missing in this puzzle though: unraid's "mover" and the ability to use a NVME disk as "cache" for all my data.
Could you make some recommendations/documentation discussing "tiered storage" model for PMS? I think many people would want it.
Unraid's ability to have FIFO writes to a NVME then later offloaded onto spinning hard drives (with spun-down settings to lower power consumption) is a major feature for home media server users. Snapraid (as you suggested) fixes many flaws and gaps that unraid model has in term of data assurance/checksumming.
I think a blog post and configuration examples with a "mover like script" to take care of transitioning data off NVME back onto the archive/spinning-rust disks would be great. While the author of mergerfs @trapexit has some information on this (https://github.com/trapexit/mergerfs#tiered-caching); a working example on your blog I think would greatly help others trying to learn many new things at once to replace unraid.
You don't support RSS/atom for feed crawling?
Why not? #living-the-rss-life
Is Docker installed on the host itself or inside a Proxmox VM/unprivileged LXC? I found this article on ServeTheHome that deemed the former method not recommend for production. See also this Reddit post.
Running on the host itself makes access to the MergerFS pool a lot simpler, but I’m worried about the security issues some Reddit users brought up. Running inside KVM requires an NFS share which results in extra overhead (related to #6). Running inside LXC does allow for host directory mounts but the storage driver needs to be changed to overlay(2). I’m curious what your current setup looks like.
As a side note, I’m currently in the process of setting up Proxmox + Docker + MergerFS + SnapRaid using your guides as a starting point for my perfect home media server. If you want, I can contribute with PR’s containing my findings along the way.
I just finished reading the section ZFS with MergerFS and from my understanding it is not clear where the snapraid parity data for the mergerfs disks is actually stored. In the diagram for the snapraid configuration is a extra parity disk. This disk is missing in the diagram for the zfs mergerfs combined configuration. This is confusing.
Is it stored on the mirrored zfs pool or is stored on 1 of the 5 disks or is stored on a unmentioned disk 6?
MergerFS also does not provide mitigation for drive failure, this is where SnapRAID comes in. It is used to calculate a snapshot of parity across all data drives that are not using ZFS. The ZFS drives are configured in a mirror and take care of their own parity and data integrity checks.
Something like this:
The TiddlyWiki container linked in your top 10 is sadly no longer maintained. The advertised fork also appears unmaintained (last updated in April 2023).
https://github.com/djmaze/tiddlywiki-docker
https://github.com/m0wer/tiddlywiki-docker
The link under About points to soon.perfectmediaserver.com
No issues, just wanted to say thanks for your time and effort put into this , on the other hand my wallet and wife wont be very happy 😂
When sharing PMS links via social means, the preview is rather mundane but mostly problematic: It's pixelated. Or doesn't exist at all..
I'd like to argue this type of preview can be much improved.
Here I think it's just a stretched favicon?
Preview could look as fancy as:
link to click/share for testing: https://perfectmediaserver.com/06-hardware/new-drive-burnin/
I just checked Ubuntu Server 20.04.2 and it contains the previously missing devices of /dev/disk/by-id/ata-*
, but also kept the scsi-*
ones.
nlz@chaos:/mnt$ ls -l /dev/disk/by-id/ | grep sdc$
lrwxrwxrwx 1 root root 9 Jun 30 13:36 ata-ST4000VN008-2DR166_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root 9 Jun 30 13:36 scsi-0ATA_ST4000VN008-2DR1_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root 9 Jun 30 13:36 scsi-1ATA_ST4000VN008-2DR166_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root 9 Jun 30 13:36 scsi-35000c500b27d88be -> ../../sdc
lrwxrwxrwx 1 root root 9 Jun 30 13:36 scsi-SATA_ST4000VN008-2DR1_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root 9 Jun 30 13:36 wwn-0x5000c500b27d88be -> ../../sdc
So maybe the advice against Ubuntu Server could be downgraded into a warning that Canonical likes to mess stuff up.
How are you setting up your storage / disks when you moved to proxmox?
Great stuff, thanks!
Maybe I'll write a PR at some point, but for now just generic thoughts:
I suppose you meant begin
.
https://github.com/IronicBadger/pms-wiki/blame/main/docs/installation/manual-install.md#L331
Hi Alex,
Great guide, thanks for your work.
The link on docker-compose page to the bash alias section on the quality of life tweaks page is broken.
It appears that the list of bash aliases on the first page is not duplicated on the second, and there is not a corresponding #bash-alias
heading or tag in the second page.
First of all, thank you for your Traefiki 101 guide—it got me up and running after a lot of searching. However, I ran into one hiccup, and that was not specifying the resolvers here like you do in your infra repo. Once I included them I was up and running, so maybe they should be included in your bare minimum setup. Thanks again, love all you do!
Images on docs/01-overview/nas-software-comparison.md
does not show on the live site, they do however show in the markdown viewer in github, suggesting funny paths
Love this project, Been using it for a couple years now after my xigmaNAS setup was no longer covering my needs.
Now that I have been running for a while I am looking to increase my pms capacity.
I am considering the following.
The SnapRAID section of the Wiki states that SnapRAID isn't suitable for real-time use cases, and users should consider an alternative. However the page makes no mention of any alternatives. I've did some initial Googling and found no good solutions as well. For a use-case like running daily backups to a NAS or running a database, what real-time alternatives to SnapRAID are there?
The ZFS link on the home page is broken. My assumption link should point to https://perfectmediaserver.com/tech-stack/zfs/
I was wondering if you had any info on using/setting Backblaze or IDrive to backup your pms?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.