Category: Linux Scripts

Posted on

WordPress database migration from hell

I finally managed to migrate my wordpress DB to the new server after several weeks of making multiple db dumps and restorations which would fail to load all the content from the old DB. I tried different tools that would export the DB as sql files and used same tools to import even straight command line dumps, but somehow my posts would not display correctly the titles and dates would display but not the post body content.

While trying to trouble shoot the cause I opened the sql file and started running each tables insert query one at a time as you can expect this sucked really bad, but I was ready to do it this way as nothing else was working. When I tried to paste the insert statements for the posts table something weird happened only one character would paste into the sql console which was weird as I expected the several hundred lines I had copied to show up. To look at this some more I posted the text in an editor that could format sql and half way down the file the SQL highlighting changed and displayed half the post as text. This normally happens when there is a character that isn’t escaped properly, it also explained why the backup imports would fail. The weird part with the backup failure is none of the tools would throw an error as you would expect if there was an error with the insert queries. While the curious part of me wanted to investigate this further the rest of me was tired with several weeks of trying to migrate the wordpress data and tried to find another way of moving the data without having to export to SQL files.

The solution I went with was DBeaver which has a data export method of type database which will let you copy a table from one DB to another without having to rely on SQL dumps. This worked like a charm even though it still took a good 10 minutes to copy over everything and keep validating as I went. Like most sites I have an DB backup scripts but since it looks like the backups created in this method may not be of use if I needed them in the future am thinking of just having another offline DB and keep that synced with my main db and use the offline DB for any restorations I might need.

Posted on

Automate letsencrypt certificate renewal

I recently switched from self signed certs to free SSL certs from letsencrypt and for the first time I could load my https url without getting the annoying prompt from chrome due to self signed certificates. The only problem is the certs expire pretty fast in about 90 days as of this writing, while this is nothing to complain about since the certs are free handling the renewal each time manually would be a pain and also leave me in a bind in case I forgot to do it.
I decided to automate the renewal process to save myself the hassle of having to do it manually and found two resources here and here on how to do it, I went with a combination of the two methods as my requirements were different.
I wanted the renewal to be run from a script to support email notification on success or failures which is similar to the first source and use the webroot plugin to perform renewal as it has lesser steps to perform renewal reducing any failure points during the process like the second source. The script needed to be able to run everyday and check cert expiration I didn’t want to hard code the cron job to run based on how long the certs are valid that way if letsencrypt changes the life of the certs no change is required on my side.
Let’s get started I won’t cover the install as that’s covered by letsencrypt site, I would advise you to read the different install methods and choose the one that best fits your needs.
After performing the install
Create your config file which will contain the arguments submitted to letsencrypt api I named mine “muthii.com.ini”

rsa-key-size = 4096
server = https://acme-v01.api.letsencrypt.org/directory
text = True
authenticator = webroot
agree-tos = True
renew-by-default = True
email = root@domain.com
webroot-path = /your/webserver/path

Run the command used to create/renew your certs, which creates the certs for you and shows you the path to find them.
[cc lang=”bash”]/root/.local/share/letsencrypt/bin/letsencrypt -c /path/muthii.com.ini -d muthii.com -d www.muthii.com auth[/cc]

Only run the above command if you haven’t created your certs or are ready to renew your current certs, otherwise just grab the script file and add it to your cron. Make sure to change the emails and file paths based on your setup. I have commented out the echo statements and only enable then for testing

For someone doing this for the first time locate your ssl.conf file used by your server and set the paths to the new certs

SSLCertificateFile /etc/letsencrypt/live/domain.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/domain.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/domain.com/fullchain.pem

Once you are done setting up head over to SSLLabs and test your certificate is recognized as expected, then setup a cron job to run the script daily .

0 2 * * * sh /path/SSLRenew.sh

Posted on

Owncloud Error: Console has to be executed with the same user as the web server is operated

After a recent Owncloud 8.o.x update I started getting this error being logged whenever the owncloud cron job run. To resolve the issue I had to change the cron job to be run as the user apache.

su -s /bin/sh apache -c "php -f /path/to/owncloud/cron.php"

The webserver on CentOS is run under the user apache, on other linux flavous it’s www-data to find out what it is on your system just check the error being logged it will log the user running the webserver.


Console has to be executed with the same user as the web server is operated
Current user: someuser
Web server user: apache <- This is the user you want. Unexpected error!

Posted on

Love OOP Python

Python makes it easy to automate manual tasks. Over time I have accumulated some scripts each for a different task but not sharing any code between them but having some similar logic, basically a few of them were having duplicate logic. So this week as I was adding another script to download and add the latest plex media server to my local repo, I came over an article about OOP python on Stackoverflow.
After creating a class for all the common logic in my scripts I was able to reduce the size of most of them by over half, I make my living writing OOP code for java but OOP for python was new to me. I wish there was OOP for bash šŸ™‚ I have a bunch of bash scripts that need culling.

Posted on

Update enom dynamic IP address

I have been using LqConsulting for about five years now to register my domain, if you are a linux user and you have used Linuxquestions to get some issues resolved consider using this registrar as it it is owned by the same person.

Back to the point this registrar currently uses enom to register your domains, which offers DyDns services and you can get a list from both places on the clients you can use to update your Dynamic IP. The problem comes with configuration, I like most people who rum linux to host personal websites have never setup one of this Dydns clients before and the help I got from my registrar was not very helpful. So for sometime as you can see from this post I was using a script to notify me when my ip changed and i would log in to my registrar and update my IP manually, this is not a very ideal way of doing it. Am not trying to bash my registrar here they provide me with a great service, they are a small shop and the main part is they do a lot for linux users, this being a personal site I was not loosing any sleep or money due to my site going down as a result of my IP changing.

But life is about convenience so I finally crawled the net and was able to find configuration info for ddclient with enom services, straight away I setup ddclient in my Centos box, and life was great anytime my ip changed I would get an email notification and my registrar would be updated automatically. The problem with life is once you have gone forward it’s hard to go back so when I updated my Centos box to the next version it came with a new version of ddclient which would not take the patch to make it work with enom but it also did not work with enom even though it was supposed to support it. So I could go back to manually updating my IP or get another client that would work with enom, I ended up putting together this python script to update my IP changes and to also send me an email whenever my dynamic IP changed. This is my first ever python code, I have worked with other languages and it was a real pleasure to see how simple python makes some tasks that would take a lot more coding in other languages I have used. Most of the code is made up of different pieces i found around the net so I can’t claim to have done it all by myself. So in the spirit of others before me am putting it out there to help anyone else who might need it feel free to use as you please and change it to your liking. If it works for you or not let me know but it is not a requirement.

Posted on

Upgrading CentOS

Do a back up a recoverable system backup or a backup for all important docs you have on system before you proceed with the commands below

yum clean all
yum update glibc\*
yum update yum\* rpm\* python\*
yum clean all
yum update
reboot

CheckĀ  yourĀ  new version with :

lsb_releaseĀ  -a

Posted on

IP change notification by email

I had installed ipupdate while trying out dnsexit.com and loved its simplicity, dnsexit.com provides a great service but in the end I chose to stick with my current registrar lqconsulting who is a major supporter of Linux users, they also run linuxquestions.org of which am a member. I had tried to configure ddclient to work with my registrar but I did not have enough info about my registrar’s requirements of the client to work with their site. So in the mean time I have edited ipupdate to send me an email everytime my ip address changes while still doing evrything else it does. I prefer it this way over updating my registrar, since I do a lot of remoting in and it takes a few minutes before a new IP is updated to nameservers “around 20min”, but if I already have my new IP in my mail when it changes I don’t have to wait until nameservers update my site IP for me to be able to remote in. I have provided the edited file below for anyone who wants to use it also the tar,rpm and deb files that install it.

For install instructions go to:

My edited file
You can replace the installations ipupdate.pl with mine for v1.6-2 or just copy over my additions to them whichever works for you.

Posted on

Find files larger than * in linux

I found this neat cmd to use when trying to find files larger than certain sizes e.g

find / -type f -size +200M

If you want files less than that size you can do

find / -type f -size -200M

For other ways to use the cmd check out this post where i got the info from.

This other cmd finds files larger than 10M and calculates their sizes too.

find / -size +10240000c -exec du -h {} \;

Remember in all cases to change “/” to the location you want to search for the files e.g “/home/username”.

Posted on

Remove spaces from filenames

Removing spaces from filelemanes can be archived using various methods. The command below uses sed to do this, before running it make sure you are in the folder containing the files whose names you want to change. 

for file in *; do mv "$file" `echo $file | sed -e 's/  */_/g' -e 's/_-_/-/g'`; done