Category: server

Posted on

Setup Couchpotato with virtualenv

I run Couchpotato on my media server and since the project isn’t actively maintained it doesn’t run on python3. My linux os dropped having pip2.7 available by default likely to encourage people to migrate to python3, I decided to run it in a python virtual environment to break the apps dependency on the system python support. Below are the steps I took to get it running.

Setup virtualenv

Python3 comes with venv or creating virtual enviroments but venv doesn’t support creating environments with python2.7. Current versions of virtualenv also don’t support python2.7 so we have to install a version that has this support we will go with version 20.21.1

pip install virtualenv==20.21.1

Create your virtual environment directory I created this inside the couchpotato project but you can also place it outside if you intend to use it with more than one project.

mkdir couchpotatoserver/.venv/

Point the virtualenvironment to the actual path of your python2.7 binary

virtualenv --python=/usr/bin/python2.7 /<path>/couchpotatoserver/.venv/

This will install python2.7 and a compatible pip version in the virtual environment.

Use virtualenv

You should be able to install all your needed dependencies from the couchpotato requirements file with the pip cmd. Use the path to the pip binary to make sure the dependencies get installed in your virtual environment and not the OS.

.venv/bin/pip install -r requirements-dev.txt

Just as we had to install a specific version of virtualenv which supports Python2.7 we will need to install specific versions of 2 other libraries needed by couchpotato in our virtual environment.

.venv/bin/pip install cryptography==2.2.2
.venv/bin/pip install PyOpenSSL==17.5.0

Update whichever script you are using to startup couch potato to use the Python installed in the virtual environment, it is very important to reference the actual path otherwise by omitting the path you will be using the system level Python version which will cause startup to fail.

/<root-path>/couchpotatoserver.venv/bin/python /apps/couchpotatoserver/CouchPotato.py

You should now be able to startup your couchpotato service using your virtual environment and should be safe from future OS updates breaking your setup. This documentation assumes you already have an existing couchpotato setup already, the setup of couchpotato is outside scope of this notes.

Posted on

Update Remi PHP to next version

Since Centos ships with an old PHP version, there are many tutorials on how to install Remi PHP which enables you to use more modern versions of PHP on your site. But when it came time to update from PHP72 to php73/4 I couldn’t find a lot of docs on this so I decided to put this one up to remind me next time am doing this.

First, we will need to enable the PHP version we want to install such as below.

# yum-config-manager --enable remi-php73
# yum-config-manager --enable remi-php74

In my case, I was moving from php72 to 73 before 74 so I enabled 73 and disabled 72.

# yum-config-manager --enable remi-php73
# yum-config-manager --disable remi-php72

After enabling the version you want to move to and disabling your current version, run yum update and restart your webserver.

# yum update
# apachectl restart

To validate the update was successful check the current version of PHP as reported in your terminal, and those are the steps needed to move from one Remi PHP version to another.

# php -v
PHP 7.3.29 (cli) (built: Jun 29 2021 09:30:31) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.29, Copyright (c) 1998-2018 Zend Technologies
Posted on

Running KVM and docker on the same host with a bridge

I have been running kvm with a bridge on my fedora machine for a while, when trying to run docker on the same host access to the kvm hosts would die as soon as docker came up. I read up online and most places mention that the two technologies should be able to co-exist with each other without any problems, after some further searching I was able to find an article that mentioned that if you have already setup a bridge for KVM you can tell docker to use this bridge. I was able to test this on my setup and it worked allowing docker to run without interfering with the existing bridge. Looking at the interfaces I can still see docker created an interface docker0 but it doesn’t seem to be active.

The docker configuration involved adding this file /etc/docker/daemon.jsonwith the following entry.
{ "bridge": "br0" }
Make sure the bridge number matches your existing bridge number in use by kvm.

Posted on

WordPress database migration from hell

I finally managed to migrate my wordpress DB to the new server after several weeks of making multiple db dumps and restorations which would fail to load all the content from the old DB. I tried different tools that would export the DB as sql files and used same tools to import even straight command line dumps, but somehow my posts would not display correctly the titles and dates would display but not the post body content.

While trying to trouble shoot the cause I opened the sql file and started running each tables insert query one at a time as you can expect this sucked really bad, but I was ready to do it this way as nothing else was working. When I tried to paste the insert statements for the posts table something weird happened only one character would paste into the sql console which was weird as I expected the several hundred lines I had copied to show up. To look at this some more I posted the text in an editor that could format sql and half way down the file the SQL highlighting changed and displayed half the post as text. This normally happens when there is a character that isn’t escaped properly, it also explained why the backup imports would fail. The weird part with the backup failure is none of the tools would throw an error as you would expect if there was an error with the insert queries. While the curious part of me wanted to investigate this further the rest of me was tired with several weeks of trying to migrate the wordpress data and tried to find another way of moving the data without having to export to SQL files.

The solution I went with was DBeaver which has a data export method of type database which will let you copy a table from one DB to another without having to rely on SQL dumps. This worked like a charm even though it still took a good 10 minutes to copy over everything and keep validating as I went. Like most sites I have an DB backup scripts but since it looks like the backups created in this method may not be of use if I needed them in the future am thinking of just having another offline DB and keep that synced with my main db and use the offline DB for any restorations I might need.

Posted on

Automate letsencrypt certificate renewal

I recently switched from self signed certs to free SSL certs from letsencrypt and for the first time I could load my https url without getting the annoying prompt from chrome due to self signed certificates. The only problem is the certs expire pretty fast in about 90 days as of this writing, while this is nothing to complain about since the certs are free handling the renewal each time manually would be a pain and also leave me in a bind in case I forgot to do it.
I decided to automate the renewal process to save myself the hassle of having to do it manually and found two resources here and here on how to do it, I went with a combination of the two methods as my requirements were different.
I wanted the renewal to be run from a script to support email notification on success or failures which is similar to the first source and use the webroot plugin to perform renewal as it has lesser steps to perform renewal reducing any failure points during the process like the second source. The script needed to be able to run everyday and check cert expiration I didn’t want to hard code the cron job to run based on how long the certs are valid that way if letsencrypt changes the life of the certs no change is required on my side.
Let’s get started I won’t cover the install as that’s covered by letsencrypt site, I would advise you to read the different install methods and choose the one that best fits your needs.
After performing the install
Create your config file which will contain the arguments submitted to letsencrypt api I named mine “muthii.com.ini”

rsa-key-size = 4096
server = https://acme-v01.api.letsencrypt.org/directory
text = True
authenticator = webroot
agree-tos = True
renew-by-default = True
email = root@domain.com
webroot-path = /your/webserver/path

Run the command used to create/renew your certs, which creates the certs for you and shows you the path to find them.
[cc lang=”bash”]/root/.local/share/letsencrypt/bin/letsencrypt -c /path/muthii.com.ini -d muthii.com -d www.muthii.com auth[/cc]

Only run the above command if you haven’t created your certs or are ready to renew your current certs, otherwise just grab the script file and add it to your cron. Make sure to change the emails and file paths based on your setup. I have commented out the echo statements and only enable then for testing

For someone doing this for the first time locate your ssl.conf file used by your server and set the paths to the new certs

SSLCertificateFile /etc/letsencrypt/live/domain.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/domain.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/domain.com/fullchain.pem

Once you are done setting up head over to SSLLabs and test your certificate is recognized as expected, then setup a cron job to run the script daily .

0 2 * * * sh /path/SSLRenew.sh

Posted on

CIFS VFS: cifs_mount failed w/return code = -2

I hit this error while adding a samba mount to my fstab, but mounting the same end point would work when executed from command line. For my scenario it turns out that it might be an issue with cifs-utils or kernel if your mount point is under more than one sub-directory. My solution was to go with option 3 and have my target as a share


"//host.IPAddress/share/subdir/subdir/target" - This failed with error "CIFS VFS: cifs_mount failed w/return code = -2"
"//host.IPAddress/share/target" - This worked
"//host.IPAddress/target" - This worked

Posted on

Cloud heaven = Plex + Android + CloudSync + Linux server + Own Cloud

I recently started hosting my own cloud service and it has been an awesome experience. After having my smart phone for sometime i started running low on storage, while I had backed up some pictures to a local share this is not really convenient. We take pictures on our phones so that we can have them close by so having to be at home to view my photos and videos was not very interesting. With a few apps and a Linux server I was able to put together a cloud service for the whole family that we are all loving very much.

The setup
Own Cloud installed on the Linux server. I tried a couple own cloud plugins for multimedia and none worked out for me. Using own cloud’s android sync app I ran into issues as it would lock up my phone when syncing. To fix this two issues I decided to have Plex handle the multimedia stuff and CloudSync handle the syncing process. Since I already had Plex serving up my video content adding photos was a breeze, plex also displays all your photos as thumbnails on your phone which you can click to view while the own cloud app displays a file name list which you have to click on a file to view it. I installed cloudSync on all the phones that needed backing up, this is like the swiss knife of sync apps it supports several cloud services plus regular services such as sftp,ftp and webdav just to name a few. You can back up your data to different locations by setting up filters for each data type which is awesome as I don’t want pictures and videos in one folder.
With this setup am able to have pictures and videos synced instantly and rsynced between different storage clients for redundancy. Within minutes of any of any of the phones taking a picture or a video any of the other phones can view that content so no need to send each other whatever you just captured.

While regular cloud providers offer you a few gigs for free and you have to pay once you get over 20gigs by hosting your own cloud you have terabytes of data readily accessible under your own control. You can use this cloud providers for just backing up your sensitive documents that you need an offsite backup in case of fire etc but remember to encrypt it first.

So now with everything backed up I have cleared my phones storage barely using a quarter of it and I still have access to all the pictures and clips that have been taken by any phone in the house for the last few years.

Posted on

Update enom dynamic IP address

I have been using LqConsulting for about five years now to register my domain, if you are a linux user and you have used Linuxquestions to get some issues resolved consider using this registrar as it it is owned by the same person.

Back to the point this registrar currently uses enom to register your domains, which offers DyDns services and you can get a list from both places on the clients you can use to update your Dynamic IP. The problem comes with configuration, I like most people who rum linux to host personal websites have never setup one of this Dydns clients before and the help I got from my registrar was not very helpful. So for sometime as you can see from this post I was using a script to notify me when my ip changed and i would log in to my registrar and update my IP manually, this is not a very ideal way of doing it. Am not trying to bash my registrar here they provide me with a great service, they are a small shop and the main part is they do a lot for linux users, this being a personal site I was not loosing any sleep or money due to my site going down as a result of my IP changing.

But life is about convenience so I finally crawled the net and was able to find configuration info for ddclient with enom services, straight away I setup ddclient in my Centos box, and life was great anytime my ip changed I would get an email notification and my registrar would be updated automatically. The problem with life is once you have gone forward it’s hard to go back so when I updated my Centos box to the next version it came with a new version of ddclient which would not take the patch to make it work with enom but it also did not work with enom even though it was supposed to support it. So I could go back to manually updating my IP or get another client that would work with enom, I ended up putting together this python script to update my IP changes and to also send me an email whenever my dynamic IP changed. This is my first ever python code, I have worked with other languages and it was a real pleasure to see how simple python makes some tasks that would take a lot more coding in other languages I have used. Most of the code is made up of different pieces i found around the net so I can’t claim to have done it all by myself. So in the spirit of others before me am putting it out there to help anyone else who might need it feel free to use as you please and change it to your liking. If it works for you or not let me know but it is not a requirement.

Posted on

Great Linux CMDs

In this post I will be making a list of great Linux cmd’s that I find all over the net one great place to stop by is commandlinefu.com they have a great deal of important linux cmd’s.

First cmd is mtr, it combines your ping and netstat cmd’s. In CentOS you have to su- to run this cmd:

mtr google.com

The following cmd I just started using recently and it has been a life saver I don’t know how I went all that time without it. The cmd is screen it works like your VNC or NX but for the terminal, it is especially handy if you SSH into your box but you don’t want to leave the terminal running waiting for a task to finish, you just start a screen run your cmd and detach from the screen. Now even if your connection gets disconnected or you disconnect yourself all you have to do when you connect again is run “screen -dR” and you are connected back to your earlier session and you can check on the progress of your task. To list screen instances “screen -ls”.

screen

Ctrl+a   ->pressing this twice moves to the next screen

Ctrl+a+c  ->adds a new screen to you screen instance

Ctrl+a+d ->detach from screen session

Ctrl+d  ->exit a screen

screen -dR   ->connect to/start a screen session

Mount a drive world writeable

mount -t ntfs-3g -o umask=000 /dev/drive /mnt/mountpoint

mount -t cifs -o umask=000,username=username,password=password //ipaddress/folder /mnt/mountpoint

by adding the option umask=000 any user can write to the mounted folder, this comes in handy when you want to write to a mounted drive as a regular user.  I have had a problem with some slack based distro’s not accepting this option in the current format and might need to play around with the mask cmd placement to see what works.

Posted on

IP change notification by email

I had installed ipupdate while trying out dnsexit.com and loved its simplicity, dnsexit.com provides a great service but in the end I chose to stick with my current registrar lqconsulting who is a major supporter of Linux users, they also run linuxquestions.org of which am a member. I had tried to configure ddclient to work with my registrar but I did not have enough info about my registrar’s requirements of the client to work with their site. So in the mean time I have edited ipupdate to send me an email everytime my ip address changes while still doing evrything else it does. I prefer it this way over updating my registrar, since I do a lot of remoting in and it takes a few minutes before a new IP is updated to nameservers “around 20min”, but if I already have my new IP in my mail when it changes I don’t have to wait until nameservers update my site IP for me to be able to remote in. I have provided the edited file below for anyone who wants to use it also the tar,rpm and deb files that install it.

For install instructions go to:

My edited file
You can replace the installations ipupdate.pl with mine for v1.6-2 or just copy over my additions to them whichever works for you.