MasonSQL::Report

Revision

  $Revision:  $

Description

MasonSQL implementation of reporting.

Included Libraries and Subroutines

  Library or Function Name    | Note
  ---------------------------------------------
  Encode                      |
  HTML::Entities              |
#  Moose                      | Used to develop.
  Mouse                       | Fast replacement
                              | of Moose.
  DBI                         |
  ODF::lpOD                   |
  Image::Info                 |
  File::MimeInfo              |
  DTL::Fast                   |
  File::Temp qw/ tempfile /   |
  Cwd qw(realpath)            |
  Text::Balanced              |
  Data::Dumper                |
  IPC::Run qw(run)            |
  IPC::Run::SafeHandles       |
  feature qw(say)             |

Private Variables, Global to Report Object

  Variable Name       | Description
  ------------------------------------------------------------------------
  $dbh                | Database handle.
  %cli                | A key-value pairs of reserved cli options.
  @reserved           | A names of the reserved cli options.
  %cmd_parameters     | A key-value pairs of command line cli options.
  %config             | A key-value pairs of configuration variables from 
                      | the Apache config file.
  $odt_rpt_id         | A primary key value of the selected report from 
                      | the public.odt_reports table.
  $dtl_context        | A DTL context which is global to class.
  %portion            | A structure containing public.odt_report_portion 
                      | records for the selected report.
  %content            | A portion ID-content pairs, that are extracted from
                      | a template file.
  @appendable_seq     | An array of portion IDs that can be inserted into report,
                      | sorted by ord field.
                      | Insertable portions are of type odt_section,
                      | odt_table, odt_file or pdf_report.
  @attachable_seq     | An array of portion IDs that can be attached as a file,
                      | sorted by ord field. 
                      | Attachable portions are of type pdf_file or pdf_report.
  $base_id            | A base template portion ID.
  $base_file_name     | A base template file name.

Private Static Variables

Temporary Image Filepaths

  Variable Name       | Description
  ------------------------------------------------------------------------
  @tmp_image_filepath | A list of filepaths to temporary files that get deleted
                      | during destruction of Report object.

push_tmp_image()

Add a new temporary image filepath to the tmp_image_filepath.

unlink_tmp_images()

Unlink each temporary image filepath from the tmp_image_filepath.

Hash of Pictures

  Variable Name       | Description
  ------------------------------------------------------------------------
  %pictures           | A hash of images added to the C<Pictures> subdirectory in
                      | the B<.odt> file. It helps detect and eliminate repeated
                      | updates of duplicates.

is_in_pictures()

If the image file name was already updated returns 1. Otherwise returns 0 and saves the image name to the pictures hash.

Interface Subroutines

parse_cli()

Parse input arguments, for example:

  $rpt->parse_cli(\@ARGV);

First check if an option is reserved otherwise check if its cmd parameter.

Reserved options contain : separator, cmd parameters contain = separator.

parse_config()

Parse Apache configuration file. The config parameters are saved to the config hash.

Currently the filepath to the config file is obtained from the CLI option config_path.

connect_to_db()

Connect to the PostgreSQL database and set the database handle.

The ReportDBIconnect variable is obtained from the Apache config file.

The connection is configured to:

  $dbh->{pg_enable_utf8} = 1;
  $dbh->{RaiseError} = 1;
  $dbh->{ShowErrorStatement} = 1;
  $dbh->{'pg_server_prepare'} = 0;    
  $dbh->do(q{
    set client_encoding TO 'UTF-8';
    set DateStyle to 'SQL, EUROPEAN';
    });

get_odt_report()

Gets selected record from public.odt_reports table. From the table it extracts odt_rpt_id and global_parameters field. The global_parameters field is parsed resulting parameters are stored into cmd_parameters hash. Also the reference to %cmd_parameters hash is added to DTL context as ARGV thus the resulting parameters are rendered as {{ ARGV.param_name }}.

get_odt_report_portions()

Gets records from public.odt_report_portions table belonging to the selected report from the public.odt_reports table. The records with empty type field are skipped. Fields from each obtained record are stored into portion structure. If the record is base report base_id and base_file_name are also obtained. It is also determined whether the portions is appendable or attachable. When the base report query is defined, it retrieves the first row of the query results, then adds the base portion name with field values from the row to the DTL DTL context.

make_report()

Makes the .odt report document.

The procedure to make a report document:

1. Create new report from the base template file.
2. Read portions from the base template file.
3. Delete portions from newly created report file.
4. Obtain sequence in which the portions will be added to the report.
5. Generate body of the report.
6. Finally save the report.

get_filepaths()

Returns the filepaths that are needed by Unoconv for conversion from .odt report to the .pdf.

The filepaths originate from the report.pdf component where they are composed.

add_attachments()

Concatenates the report with the attachments. The attachments are ordered by attachable_seq

The concatenation is made with pdftk executable, with the command:

  pdftk <filepath1> <filepath2> ... <filepathN> cat output <output_filepath>

Parsing CLI and Apache Config

parse_reserved()

Checks if the CLI option is found in cli hash and, in case it does, returns 1; Otherwise returns 0.

parse_cmd_parameter()

Checks that the input CLI option is a cmd parameter If its not a cmd parameter raises an error.

list2cmd_parameters()

Uses Text::Balanced to extract cmd parameters from the input list. A cmd parameters without values will cause an error unless the value is empty string. The parsed cmd parameter names and values are stored in the cmd_parameters hash.

check_mandatory_option()

Verifies that the mandatory option is reserved and that the option exists in cli hash. Otherwise raises an error.

Handling Portions

append_portion()

1. Clones the content of the template portion;
2. Renders the cloned portion;
3. Appends it to the body of the document.

attach_portion()

• If the portion type is pdf_file it constructs filepath with rendered portion's file name then attaches the filepath to the pdftk command.
• If the portion type is pdf_report it makes the pdf report then attaches the report's filepath to the pdftk command.

delete_portions()

Deletes all of the sections and all of the tables from the body of the current document.

get_portion()

Checks the portion type then, accordingly to type, uses get_section or get_table_by_name. If the portion type does not match it raises an error although the table or the section exists. If the portion type is unknown it is skipped. Since the template can be different from base template the text spans and comments are preventively removed.

update_portion()

Renders file_name then checks whether to update the content hash or skip the update. The update is skipped if the file_name is NULL, if its a base filename, if its empty string or if the template portion is already stored in the content hash. The subroutine allways returns current content key.

read_base_portions()

The subroutine is called in the initial phase of the report creation: Reads sections and tables into content hash.

set_base_context()

If its possible, adds the first row of the base portion to the DTL context. The prerequisite is availability of the query and name field values for the base portion and the row number set to 1.

set_portion_context()

Adds the portion name with field values from record to the DTL DTL context. Previous entry in the DTL context is overwritten with the current one. The added field values are used during rendering of the portion's DTL tags. The corresponding DTL reference can be used like this: {{ my_portion_name.my_field_name }} In case the field is of json type the content of that field is converted to JSON tree and stored to the DTL context. For example: The property of the JSON string {"X": {"Y": {"Z": "DATA"}}} can be accessed with the DTL tag {{ myportionname.myfieldname.X.Y.Z }}.

Handling odt_table Portions

make_table()

Uses the following procedure to create a new table:

1. Render obj_ref field value.
2. Update content of current portion.
3. Clone the template table and append the new table to the body of the report. Use the last row from the template table as a template row. If the table doesn't have any rows skip to step 6.
4. Render non-repeating table header.
5. Populate the table with the query results:

   a.

   Clone the template row and append it to the new table.

   b.

   Updates images in the current row.

   c.

   Update DTL context with the query results for the current row and with the current row number. Render the appended row.

   d.

   Check if any children portions can be added.

6. Render repeating table header.

render_repeating_header()

Gets table header and renders its content. Also updates images.

render_non_repeating_header()

Renders non-header rows except the last one which is reserved as a template row.

render_row()

Renders text in each cell in the row, consecutively one by one.

Handling odt_section Portions

make_section()

Makes a query.

For each row from the query result:

1. Updates DTL context with the values from the current row and with the current row number.
2. Appends a new section.

If there is no query it still appends one section.

add_section()

Uses the rendered obj_ref field value to find the section that is then appended to the body of the report document. Before the section is appended it is updated. In case the portion type is odt_file the .odt file is appended to the body of the report instead of a section. After the section is appended its images are updated.

Handling odt_file Portions

append_odt_file()

Renders the content of the template .odt file then appends the file to the body of the report document.

Text Processing

has_text_fields()

Determines whether the object has children text fields. Returns 1 if children fields are found, otherwise returns 0.

get_text()

Returns decoded result of get_text.

set_text()

Decodes entities from the input text then encodes the text before calling set_text

remove_spans_from_list()

Uses the recomended get_text, set_text procedure to remove entities from input list of text elements. In case the text element has any text fields children the procedure is skipped, otherwise the children text fields would be deleted.

remove_text_spans()

Removes text spans from the entire document part. The text elements are obtained with get_headings and get_paragraphs subroutines.

remove_comments()

Removes all comments from the report document.

truncate_blanks()

Removes white spaces from the beginning and from the end of the string.

DTL Rendering

render_string()

Uses DTL to render an input string.

render_text()

Renders all object's headings and paragraphs.

render_list()

Renders text from each element in the input list. In case the element has children text fields the rendering is skipped.

render_header_footer()

Accesses the page styles; Renders text elements found in page header and page footer.

Rendering Images

delete_old_images()

Deletes any previously existing image linked to the frame.

render_image()

Replaces the image with the content of its DTL tag.

Rendering steps:

1. Store the image content to a temporary file. If the image is already added to the Pictures directory inside .odt report file, skip all following steps.
2. Detect the image mime type.
3. Add the temporary image file to the Pictures directory.
4. Delete template images from the Pictures directory.
5. Add the temporary image file to the tmp_image_filepath list, to be deleted when object gets destroyed.
6. Link newly created image to the existing frame. The frame size doesn't change.

process_image()

Adds the image to the Pictures directory inside the .odt report file. A name of the new image is __. . If the DTL tag is empty or the ROW_NUMBER is missing (indicates empty query) the subroutine simply removes the image frame. After initial processing of the image the image is rendered.

update_images()

Updates each image, in object's headers and paragraphs, which contains a DTL tag in its name. The DTL tag must be in a #. form.

Ordering

get_appendable_sequence()

Sorts the appendable portion IDs into appendable_seq list, ordered according to ord field.

get_attachable_sequence()

Sorts the attachable portion IDs into attachable_seq list, ordered according to ord field.

Determing Inheritance

find_children()

Returns an ordered list of portion IDs of children portions. It returns only portions that can be appended to the body of the report document.

Making Report

make_body()

Is a recursive subroutine. It is initally called from make_report when the father ID is the base portion ID.

If first finds sibling portions then for each siblings then:

1. Renders portion's query string.
2. Makes the query.

3. a.

   If the portion type is odt_section or odt_file makes a new section.

   b.

   If the portion type is odt_table makes a new table.

The subroutine is recursively called from make_table or from add_section.

new_report()

Creates a new report.

The procedure to create a new report:

1. Open template file.
2. Save template file as a new repor.
3. Remove text spans and comments from the entire document.
4. Update images.
5. Render header and footer of the new report document.

make_pdf_report()

Make pdf report.

The procedure to make a pdf report:

1. Create a new temporary file.
2. Create a new report.
3. Delete portions from the body of the report.
4. Render the query string
5. Query the database then use the first row from the query result. Update DTL context with the values from the first row and with the row number set to 1.
6. Render the object reference field then update portion content.
7. Append portion to the newly created report.
8. Updates portion images.
9. Save the newly created report.
10. Use Unoconv to convert the newly created report to .pdf file format.

Accessing Document

get_bodypart()

Gets body of the required part of the document.

Accessing File System

unlink_files()

Unlinks files from the input list of filepaths.

template_filepath()

Constructs the filepath of a template file. The root directory for the template files of the specific report is: /public/odt_reports// directory. A file name of the template file must contain relative path to the template root directory.

The subroutine also validates that:

• The checked filepath exists.
• The checked file is not empty.
• The file is accessable for reading.

tmp_file()

Uses File::Temp::tempfile to create a new temporary file. Closes the file, to make sure it is unlocked, then returns its filepath.

Error Handling

err_msg()

The subroutine formats every die message of this module so their detection is made easier.

Destructor

DEMOLISH()

Closes down the database connection during the object's destruction.

Copyright

  (C) 2003 Leader.IT di Guido Brugnara <http://www.leader.it>
           Strada della Pozzata, 41 - Villazzano
           38123 T R E N T O   (ITALY)

Authors

  Guido Brugnara
  Janez Stangelj