Upgrade from Ubuntu Precise VPS to Xenial

On this page:

First step: Verifying the present version on the system

# It is important to know what the present version is
lsb_release -a

# The response will be similar to the following:
#
# No LSB modules are available.
# Distributor ID: Ubuntu
# Description:    Ubuntu 12.04.5 LTS
# Release:        12.04
# Codename:       precise
#
# In this case, it is necessary to run the upgrade to Ubuntu 14_04 LTS.

# It is also important to know if the system is 32 or 64 bits:
uname -i

# It may result in 'i386' for systems that run on 32 bits, otherwise, it will result in 'X86_64'<br />

Deactivation of crontab

Before the upgrade you must disable the execution of one of the cronjobs, otherwise a warning email will be sent every minute.

This cronjob can be found in the '/etc/crontab' file. And you must comment the following line:

* * * * * root /opt/masonsql/bin/cronjobs /opt/demo_masonsql/etc/apache2.demo_masonsql.conf

You must uncomment the same line only at the end of the upgrade.

From Ubuntu 12_04 LTS (Precise Pangolin) to Ubuntu 14_04 LTS (Trustly)

In order to upgrade to Ubuntu 16_04 LTS, you need to first upgrade to Ubuntu 14_04 LTS.

It is necessary to perform a backup / snapshot before starting to perform any sort of update whatsoever.

It is also advisable to run the "screen" commands to avoid problems if there is a link interruption.

# Proceed with root privileges.
sudo su -

# Package upgrades
apt-get update
apt-get upgrade
apt-get dist-upgrade
apt-get autoremove

# The 'update-manager-core' command will be used, then verifying its existence
apt-cache policy update-manager-core

# If the following line is in the output:
# ...
# Installed: (none)
# ...
# Proceed with its installation:
apt-get install update-manager-core

# Run the utility to automatically install the system upgrade
do-release-upgrade

Problems with do-release-upgrade

With this command, an automatic system upgrade attempt will be performed to the next LTS version, to 14.04, in this case.
Thus, time will be needed for the upgrade to finish.

If a critical error is encountered, the command will be interrupted showing a summary description of the problem. You will then be able to get more details by reading the log file:

/var/log/dist-upgrade/main.log

You will need to search the line with the following: 'ERROR' towards the end of the file, which will then inform you what caused the interruption.

# An example of the error from the *do-release-upgrade* command is reported in the following line,
# showing that the package postgresql-plperl-9.1 cannot be uninstalled
# because it is in the blacklist of removals:

  2017-04-03 17:54:23,100 ERROR Dist-upgrade failed: 'Il pacchetto "postgresql-plperl-9.1" e' selezionato per la rimozione, ma e' nella blacklist per la rimozione.'

# In this case, it is necessary to remove the package with:
apt-get remove postgresql-plperl-9.1

# And only at the end of the upgrade should you install it again with the following command:
apt-get install postgresql-plperl 

If the error encountered is related to not having enough space in the disk you should try the following command to open space:

apt-get autoclean

If that is not enough, you will need to temporarily widen the space available in the VPS

From Ubuntu 14_04 LTS (Trustly) to Ubuntu 16_04 LTS (Xenial)

It is necessary to perform a backup / snapshot before starting to perform any sort of update whatsoever.

# Package upgrades
apt-get update
apt-get upgrade
apt-get dist-upgrade
apt-get autoremove

# The 'update-manager-core' command will be used, then verifying its existence
apt-cache policy update-manager-core

# If the following line is in the output:
# ...
# Installed: (none))
# ...
# # Proceed with its installation:
apt-get install update-manager-core

# Run the utility to automatically install the system upgrade
do-release-upgrade 

Refer to the paragraph 'Problems with do-release-upgrade' to have an idea on how to manage certain errors that are reported by this command.

System reboot request

Upon the conclusion of the 'do-release-upgrade' command, a message will ask you to restart the computer. But this can be done only after the following operations:

# Install the 'upstart' system that manages the init script
apt-get install upstart-sysv upstart-bin upstart

# In the */etc/rc.local* file, you need to place the following lines before the 'exit 0'
chmod 666 /dev/null
chmod 666 /dev/random /dev/urandom

# Finally, you can reboot the computer
reboot 

Syslog

• Logrotate is configured to rotate the mail.info file that does not exist. To avoid errors, you should create it instead of editing the configuration files.

touch /var/log/mail.info
chown syslog.adm /var/log/mail.info

After the reboot

# Verify if the system is really the desired version
lsb_release -a

# The response will be similar to the following:
# ...
# Description:    Ubuntu 16.04.2 LTS
# Release:        16.04
# Codename:       xenial 

Add "multiverse" repositories:

sudo apt-get install software-properties-common
sudo apt-add-repository multiverse

Reactivate the following lines that are indicated as disabled during the upgrade in the '/etc/apt/soucers.list' file, and in all files in the following directory: '/etc/apt/sources.list.d/', except for those that are from the 'webmin' source; modify the name of the distribution in their report, from 'trusty' or 'precise' to 'xenial'.

If you get a warning similar to this while using 'apt-get' follow this procedure:

# When you try a 'apt-get update' the execution finishes with the following warning
apt-get update
...
N: The file "50unattended-upgrades.ucf-dist" is ignored in the "/etc/apt/apt.conf.d/" directory because it has an invalid extension

# Erase the file that causes the warning:
rm /etc/apt/apt.conf.d/50unattended-upgrades.ucf-dist

Upgrade MasonSQL

In order for Apache 2.4 to work properly, you will need to update the contents of the following directory: '/opt/masonsql'.
For this purpose you can copy the entire directory from an existing server, or update with SVN.

cd /tmp
svn co https://www.leader.it/svn/masonsql/trunk masonsql_trunk
cd masonsql_trunk
#
cp bin/cronjobs /opt/masonsql/bin/
chown root.root /opt/masonsql/bin/cronjobs
chown 700 /opt/masonsql/bin/cronjobs
#
cp etc/* /opt/masonsql/etc/
chown root.www-data /opt/masonsql/etc/apache2.masonsql.conf
chmod 640 /opt/masonsql/etc/apache2.masonsql.conf
#
cp lib/Auth_AC/*.pm  /opt/masonsql/lib/Auth_AC/
cp lib/MasonSQL/*.pm /opt/masonsql/lib/MasonSQL/
cp lib/MasonSQL/Report/Unoconv.pm /opt/masonsql/lib/MasonSQL/Report/
#
cp report/* /opt/masonsql/report/
chown root.www-data /opt/masonsql/report/*
(cd /opt/masonsql/report/; chown www-data log repmandxp.sh www-data_home_dir)
#
cp doc/* /opt/masonsql/doc/
cp -a htdocs /opt/masonsql/
chown -R root.www-data /opt/masonsql/htdocs
#
cp -a utility /opt/masonsql/
#
rm -r /opt/masonsql/perlbrew
rm -r /tmp/masonsql_trunk

Apache 2

# To reactivate apache2, it is convenient to save the configurations used
# on your old system and reinstall the package from the server
mv /etc/apache2 /etc/apache2_old
apt-get purge apache2
apt-get install apache2

# It is necessary to disable the default site of apache
a2dissite 000-default
a2dissite default-ssl

# Create the link for the masonsql site
ln -s /opt/demo_masonsql/etc/apache2.demo_masonsql.conf /etc/apache2/sites-available/demo_masonsql.conf

# and enable the site
a2ensite demo_masonsql 

You also need to activate the necessary modules for masonsql with apache 2.4 to work properly and prepare the log directory.

# Remove and reinstall some modules, and then activate them
apt-get purge libapreq2-3 libapache2-mod-apreq2
apt-get install libapreq2-3 libapache2-mod-apreq2
a2enmod apreq2
apt-get purge libapache2-mod-perl2
apt-get install libapache2-mod-perl2
a2enmod perl

# Disable the unnecessary modules
a2dismod mpm_event
a2dismod mpm_worker

# Enable the necessary modules
a2enmod auth_form
a2enmod log_forensic
a2enmod mpm_prefork
a2enmod request
a2enmod rewrite
a2enmod session
a2enmod socache_shmcb
a2enmod ssl

# Prepare the log directory and modify the permissions
chmod 755 /var/log/apache2
mkdir /var/log/apache2/masonsql
chown root.www-data /var/log/apache2/masonsql
chmod 770 /var/log/apache2/masonsql
ln -s /var/log/apache2/masonsql /var/log/apache2/demo_masonsql

# Refresh apache2
service apache2 restart 

When apache2 restarts, there can be warnings of errors that report some unacknowledged perl modules. In this case, the procedure to follow is always the same.

# Considering the following error message as example
Can't locate String/Diff.pm in @INC (you may need to install the String::Diff module)

# It follows that the module String::Diff must be installed, and thus you should check
# if its corresponding package exists in the  repository
apt-cache search string-diff

# The response to the command may be the following:
...
libstring-diff-perl - simple diff for strings
...

# The package exists in the repository and thus it must be installed from there.
# Only in cases where it is nonexistent should it be installed from CPAN.
cpan -i String::Diff 

Also, whe restarting apache there could be syntax errors on 'Options', follow this procedure for each one.

# Considering the following error message as example
Output of config test was:
AH00526: Syntax error on line 313 of /opt/masonsql/etc/apache2.masonsql.conf:
Either all Options must start with + or -, or no Option may.
Action 'configtest' failed.

# Find the file of the error and the right line and place a plus sign '+' before 'Indexes'
Options +Indexes -MultiViews

In addition, Apache guidelines must be adapted for permissions. So please refer to the document:

• https://httpd.apache.org/docs/2.4/upgrading.html

Manually installed or updated modules and packages

The following 2 lists of packages and modules that we needed to install or manually update based on our practical experience were the following:

From repository:

apt-get install libstring-diff-perl libapache2-request-perl 
apt-get install libapache-dbi-perl libstring-random-perl libdbd-pg-perl

From CPAN:

cpan DBD::Pg
cpan Mail::Sender
cpan App::cpanminus
cpan Apache2::AuthCookie
cpan SQL::Statement
cpan HOP::Lexer
cpan SQL::Dialects::ANSI
cpan Sys::RunAlone
cpan MIME::Parser
cpan Archive::Tar::Wrapper
cpan Archive::Zip
cpan Perl6::Form
cpan HTTP::BrowserDetect
cpan IPC::Run::SafeHandles
cpan Text::Balanced
cpan Captcha::reCAPTCHA
cpan ODF::lpOD
cpan DTL::Fast
cpan Image::Info

PostgreSQL

After having removed all errors seen during the restoration of apache, web communications may probably not be possible yet: the database was not updated to the conditions needed by the apache 2.4 server.

Go to the following page: https://www.leader.it/bin/view/MasonSQL/DatabaseAccess

for the proper connection procedures to the database of the server in question through the ssh + tunneling protocol.

First, you must have the credentials of a postgresql administrator user on the server.

NOTE: Verify that there is not more than one postgresql server installed, functioning, and pending on different ports apart from the default port (port 5432).

Run the following sql commands to psql or pgAdmin 3:

CREATE TABLE public.session (
  id serial NOT NULL,
  id_anagrafiche integer,
  user_agent character varying,
  user_ip character varying(15),
  session character varying(32),
  session_time timestamp without time zone,
  previus_session_time timestamp without time zone,
  session_inactive_time integer,
  CONSTRAINT session_pkey PRIMARY KEY (id),
  CONSTRAINT session_id_anagrafiche FOREIGN KEY (id_anagrafiche)
      REFERENCES public.anagrafiche (id) MATCH SIMPLE
      ON UPDATE RESTRICT ON DELETE CASCADE
);

ALTER TABLE public.session OWNER TO postgres;
GRANT ALL ON TABLE public.session TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE public.session TO masonsql;
CREATE INDEX fki_session_id_anagrafiche ON public.session USING btree (id_anagrafiche);
CREATE UNIQUE INDEX session_idx ON public.session USING btree (session);
GRANT SELECT, UPDATE ON SEQUENCE public.session_id_seq TO masonsql;

ALTER TABLE public.anagrafiche ADD COLUMN concurrent_sessions boolean;

TRUNCATE public.recordset,public.recordset_rows;

ALTER TABLE public.recordset DROP COLUMN id_anagrafiche;

ALTER TABLE public.recordset ADD COLUMN id_session integer;

ALTER TABLE public.recordset ADD CONSTRAINT recordset_id_session FOREIGN KEY (id_session)
    REFERENCES public.session (id) MATCH SIMPLE ON UPDATE RESTRICT ON DELETE CASCADE;

ALTER TABLE public.funzioni ADD COLUMN menu_style character varying(200);
COMMENT ON COLUMN public.funzioni.menu_style IS 'Stile del menù (CSS)';

ALTER TABLE public.anagrafiche ADD COLUMN last_change_password timestamp without time zone;
COMMENT ON COLUMN public.anagrafiche.last_change_password IS 'Data dell''ultimo cambiamento della password';

CREATE TABLE public.odt_report_fields (
  id serial NOT NULL ,
  key character varying(100),
  function character varying(1024),
  id_odt_reports integer NOT NULL,
  format character varying(256),
  CONSTRAINT odt_report_fields_pkey PRIMARY KEY (id),
  CONSTRAINT odt_report_fields_id_odt_reports FOREIGN KEY (id_odt_reports)
      REFERENCES public.odt_reports (id) MATCH SIMPLE
      ON UPDATE NO ACTION ON DELETE NO ACTION
);
ALTER TABLE public.odt_report_fields OWNER TO postgres;
GRANT ALL ON TABLE public.odt_report_fields TO postgres;
GRANT SELECT ON TABLE public.odt_report_fields TO report;
GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE public.odt_report_fields TO masonsql;
GRANT SELECT, UPDATE ON SEQUENCE public.odt_report_fields_id_seq TO masonsql;

If necessary run also:

ALTER TABLE public.anagrafiche ADD COLUMN last_change_password timestamp without time zone;
COMMENT ON COLUMN public.anagrafiche.last_change_password IS 'Data dell''ultimo cambiamento della password';
ALTER TABLE public.anagrafiche ADD COLUMN no_psw_expiration boolean;
COMMENT ON COLUMN public.anagrafiche.no_psw_expiration IS 'Account senza controllo di scadenza (account di sistema)';
ALTER TABLE public.anagrafiche ADD COLUMN no_change_password boolean;
COMMENT ON COLUMN public.anagrafiche.no_change_password IS 'Cambio password disabilitato';
ALTER TABLE public.anagrafiche ADD COLUMN cell_sms character varying(20);
UPDATE anagrafiche SET no_psw_expiration = null, last_change_password = now();
CREATE OR REPLACE FUNCTION public.change_password(
    text,
    text,
    boolean)
  RETURNS boolean AS
$BODY$
  DECLARE
    user_login ALIAS FOR $1;
    new_password ALIAS FOR $2;
    force_change_password ALIAS FOR $3;
  BEGIN
    UPDATE public.anagrafiche 
    SET "password" = md5(new_password), 
        last_change_password = CASE WHEN force_change_password THEN null ELSE now() END,
        session_time = now()
    where login = user_login;
    RETURN true;
  END;
$BODY$ LANGUAGE plpgsql VOLATILE COST 100;
ALTER FUNCTION public.change_password(text, text, boolean) OWNER TO postgres;
GRANT EXECUTE ON FUNCTION public.change_password(text, text, boolean) TO postgres;
GRANT EXECUTE ON FUNCTION public.change_password(text, text, boolean) TO masonsql;
REVOKE ALL ON FUNCTION public.change_password(text, text, boolean) FROM public;

ODT Report

The ODT Report type reports must be manually updated, replacing the GROUP_ID variable with REPORT_ID in all its occurrences in the templates on the menu [Permissions] -> [ODT Reports]

Also, in the SQL queries using GROUP_ID you must delete the UNION and the corresponding query with negative GROUP_ID .

• OdtReportManagerInstallation#Libreoffice

Unoconv Daemon

• OdtReportManagerInstallation#Unoconv_Daemon

Conclusion of upgrade

Run the following command to delete fstrim, not necessary in a VPS that uses the hosts file system:

rm /etc/cron.weekly/fstrim

If the update expects the substitution of the report generation engine, you can also delete the Report Manager file and remove the wine package together with the packages that are no longer used. To this end, delete the unnecessary files and folders.

rm /opt/masonsql/report/libpq.so /opt/masonsql/report/libpq.so.5.2
rm -r /opt/masonsql/report/rpdesigner-2_9a /opt/masonsql/report/rpservercline-2_9a
rm -r /opt/masonsql/report/www-data_home_dir

apt-get purge wine wine1.4 wine1.4-amd64 wine1.4-common wine1.4-i386:i386 winetricks wine1.6
apt-get autoremove

Check Let's Encrypt SSL update, after upgrade

If Let's Encrypt SSL is used on the server, you need to check if it works, after updating it as follows: https://www.leader.it/MasonSQL/LetsencryptSSL

#Check Let's Encrypt SSL is working with
/opt/certbot/certbot-auto renew

# If there's an error in the execution try the following procedure to generate the certificate
# you must first ensure that the 80 port in your server is open to the internet
rm -r /root/.local/share/letsencrypt
rm -r /opt/certbot
mkdir /opt/certbot 
cd /opt/certbot
wget https://dl.eff.org/certbot-auto
chmod a+x ./certbot-auto
./certbot-auto run
# At request "Please enter in your domain" reply with "c"
# Test if it work
./certbot-auto renew

Error from crontab: ttf-mscorefonts-installer package

If you encounter an error in updating the ttf-mscorefonts-installer package font, it depends on the lack of the apt-transport-https package

apt-get install apt-transport-https
apt-get purge ttf-mscorefonts-installer
apt-get install ttf-mscorefonts-installer