Project

General

Profile

QSS Solver: comparison tool

This page documents how to use a custom tool to compare QSS simulations (performed by the QSS solver) sweeping arbitrary ranges of precision parameters --in particular, Tolerance and AbsTolerance.

Compilation

  • This tool is currently being developed in a private BitBucket repository. Contact Lucio Santi so as he can grant you access privileges.
  • Before attempting to compile the source code, you need to make sure that the Qt SDK is installed in your system.
    • You can use apt-get to install the qt-sdk package in a Debian/Ubuntu-like environment.
  • After installing this package and retrieving the source code, you need to:
    • Run qmake-qt4.
    • Go to the src directory and run make.
  • You can then run the application using the executable qss-comparison placed in the bin directory.

Usage

  • Make sure first that you have followed these steps to set up the QSS solver.
  • Then, you can specify which model you want to study with this tool by setting first the path to the directory where the MicroModellica source files are placed (Models Path).
  • You can add an entry for a given model and specify arbitrary ranges for the Tolerance parameter as well as the AbsTolerance parameter.
    • In particular, the range is generated by starting at the minimum value provided and incrementing by the step until the maximum is reached (note that the maximum value will be excluded).
  • For every combination of tolerance and absolute tolerance, the QSS solver will be invoked when clicking the Run Exps! button.
  • Then, the Results tab will show one entry for every run, and here you can compare CPU time used by each experiment as well as the error obtained.

Error calculation

  • For each experiment, the error is calculated using a precomputed baseline experiment, which has to be specified in the Baseline File box.
    • This file is expected to be formatted as follows: each line must start with the time currently sampled, followed by the values of the output variables, separated by tabs.
    • For the N02 example, the source code includes a Python script that, given a sample period and the end time, generates a sampled baseline using the analytic solution.
  • Next to the baseline file, the sample period to use for the experiments can be defined. It is important to use the same value that was used for the baseline, as otherwise the tool will not be able to correctly compute the error.
  • This error is the square root of the mean squared error (see meeting pages for proper definitions).

Plotting and saving results

  • The results can be saved to a file in CSV format. Each entry in the results tab will be saved with the exact same fields.
  • Also, double-clicking any of these entries will render a quick gnuplot graph showing the results of the simulation (i.e., how the output variables evolved over time).

Command line interface

  • Even though it is easier to use the tool through its GUI, a new command line interface is also available.
  • This interface also sweeps parameters geometrically and not linearly. This feature will also be included in the graphical interface later.
  • Currently, the user needs to define a configuration file to specify in different lines every argument needed to run the experiments. Each of them has to follow the format key=value. These are the keys that must be provided:
    • model: Path to the MicroModellica model.
    • method: The solver method to use (e.g., QSS3).
    • min_tolerance: Minimum value to use for the tolerance.
    • max_tolerance: Maximum value to use for the tolerance
    • tolerance_factor: The geometric factor to be used for generating the tolerance range.
    • min_abs_tolerance: Same as min_tolerance.
    • max_abs_tolerance: Same as max_tolerance.
    • abs_tolerance_factor: Same as tolerance_factor.
    • N: Population size (it can be used for defining the number of crossing planes in our example).
    • baseline: Path to the baseline file to use when calculating the error (see above for further details).
    • period: The sample period with which the baseline was generated.
  • It has to be invoked like so:
    src/scripts/run_tests.py <path to config file>