Pytest-HTML release 1.10.0: Dealing with CSP
The main topic of the version 1.10.0 of Pytest-HTML was dealing with Content Security Policy (CSP), since the HTML reports started to look terrible on sites (such as Jenkins) where CSP was active. Before the report was something like that:
|As you may see, the table report was almost illegible.|
To handle with that, I had to solve many issues. I'll tell from the start.
Handling with CSS
The first thing that was breaking the CSP was that Pytest-HTML only had inline CSS, because when the plugin was built, we thought that could be a good idea to have all in a single document, so it could be exported more easily. The problem is that CSP doesn't allow inline CSS, so we had to create a new file to CSS and export it from the plugin, but for people who desire to have all contained in a single file, there is a new option to use with pytest-html: --self-contained-html, when that is active, pytest-html doesn't create any other file other than the HTML file.
Handling with images, text and JSON
The images had a similar problem of CSS, to be in a single file, they were compressed to base64 strings. The problem with that is that a CSP page doesn't accept to render a base64 string as an image. So the solution was similar to the one applied to CSS, there is a code which creates images on an external file, and there is an option to still creates the images alongside the HTML when it is necessary to easily share that file. There was not problems related with CSP, JSON and text, but there were an open issue suggesting that we should create then in separated files as well, so we did that.
Initial sort order of the Results
Prior, the sort was all made by JS, so when JS was disabled (CSP disallow JS), we lost the ability of sorting the table (there is a default order that results should appear by type, which is Error, Failed, Rerun, Skipped, Passed), this was also a very cost operation to use with JS when there was a big amount of tests, many times the browser used to stop working when this script was loaded. So the idea was to order that using the python plugin. Since ordering is a very cost operation, the better idea is doing an ordered insertion (every time a row was inserted in the table, it should be inserted on its right position), and for doing that, I created an object that represents the TestResult, and before inserting each test result on the table, I compared it to the others by its outcome. Now, even more, the plugin is object oriented, we have an object that represents the report, other one the TestResult and other one the Outcomes of the test set.
Minor issues: Rerun outcome and filter checkboxes
Since last release, we started to have two new things: support to rerun-failure plugin and the ability to filter the results by its outcome. The problem is that the rerun should not always appear, only when the rerun-failures is enabled, and the filter function is totally dependent on Java Script. Thanks to OOP, doing those changes were quick, I just had to add a condition before creating the rerun object, and change one line of code where the code of the checkboxes were being created inside the rerun class.
Contribution of external public: collapse and show logs
One enhancement that was asked a little time ago was a button to hide logs, and another one to show those logs, so there was an external contributor named Leitzler who did that.
That is it! And see the result of the new reports of the HTML plugin right here: https://webqa-ci.mozilla.com/https://webqa-ci.mozilla.com/