Dan Čermák
653814afb0
[testsuite] Allow setting the binary location for make tests
2018-06-07 10:19:59 +02:00
Dan Čermák
35e7037f86
[testsuite] Change the calling order of setUp & tearDown in decorator
...
The FileDecoratorBase injects a new setUp & tearDown function. These new
functions would call the old setUp & tearDown in an inconvenient order: e.g. the
child class CopyFiles would at first call the user provided setUp and then copy
the files. This makes it impossible to perform some action on the file copy in
setUp.
=> This commit changes the call order, so that setUp & tearDown always "see" the
finished environment after file copies are in place and before any cleanup took
place.
2018-06-06 13:59:45 +02:00
Dan Čermák
39b7151b2e
[testsuite] Port redmine issues 831 to 937 to the new testsuite
2018-06-06 13:59:34 +02:00
Dan Čermák
7e1cd7d1b6
[testsuite] Fix name of regression test #283
...
issue got a CVE assigned
2018-06-06 13:31:53 +02:00
Luis Diaz Mas
5bd26ee82b
Port redmine issues 1179-1305
...
- Authorship: Luis Díaz Más & Dan Čermák
2018-06-01 07:56:13 +02:00
Dan Čermák
d7f4453dbd
[testsuite] Remove usage of $cat via hooks
2018-05-29 10:21:13 +02:00
Dan Čermák
41ac25c4f4
[testsuite] Add hooks & documentation for hooks
2018-05-29 10:21:13 +02:00
Dan Čermák
23e29e9527
[testsuite] Use system_tests.path() in some test cases
2018-05-29 10:21:13 +02:00
Dan Čermák
4e32929956
[testsuite] Add notes concerning escaping in commands
2018-05-29 10:21:13 +02:00
Dan Čermák
53c265f8a7
[testsuite] Add path conversion function
2018-05-29 10:21:13 +02:00
Dan Čermák
300b1dc0ef
[testsuite] Testsuite injects config file values into its namespace
2018-05-29 10:21:13 +02:00
Luis Diaz Mas
8eca055058
Add POC + regression test.
...
Note that we override here the method check_no_ASAN_UBSAN_errors to also
analyze the last two lines of got_stderr.
2018-05-26 09:59:37 +02:00
Luis Díaz Más
b8b94dc67a
Escape characters properly once we started using python literals
2018-05-24 11:06:18 +02:00
Luis Díaz Más
48078e9c8b
Fix tests on windows by using os.path.join
2018-05-24 11:06:18 +02:00
Luis Díaz Más
72faa7331d
Fix issues with python strings & literals
2018-05-24 11:06:18 +02:00
Dan Čermák
90c8d754b7
[testsuite] Call subprocess.Popen() with shell=True on Windows
2018-05-24 11:06:18 +02:00
Dan Čermák
939ded283f
[testsuite] check stderr before stdout
...
That way you get to see error messages about wrong filenames instead of an
empty stdout
2018-05-24 11:06:18 +02:00
Dan Čermák
f55cdce761
[testsuite] Port redmine issues 498-825
2018-05-24 11:06:18 +02:00
Dan Čermák
2356984274
[testsuite] Ported redmine bugfixes 426-495
2018-05-24 11:06:18 +02:00
Luis Diaz Mas
2fb00c8a16
Analyze minimum needed number of null separators in PngChunk::parseTXTChunk
...
This commit fixes the heap-buffer-overflow in PngChunk::parseTXTChunk.
According to the specification:
http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html
There must be 2 null separators when we start to analyze the language tag.
2018-05-23 10:57:13 +02:00
Luis Diaz Mas
234e5794f6
Add tests for #306
2018-05-23 10:57:13 +02:00
Daniel Zucchetto
cb5135f772
Add test for Canon M100 ModelID
2018-05-21 23:35:39 +02:00
Robin Mills
4239ff6734
Test for Tamron SP AF 150-600mm F/5-6.3 VC USD G2 (A022) for Nikon
2018-05-06 12:00:20 +01:00
Dan Čermák
b364e075c6
[testsuite] Implement DeleteFiles decorator
...
- Refactor FileDecoratorBase to use an arbitrary property in the decorated
class where the list of files is saved
- Add DeleteFiles decorator which deletes all files passed to it after the test
finished
2018-04-25 09:41:50 +02:00
Dan Čermák
d1714f6564
[testsuite] Fix typo and obsolete syntax
2018-04-25 09:35:42 +02:00
Dan Čermák
fe98936375
[testsuite] Fixed infinite iteration in CaseMeta.__new__
...
The loop condition in the variable expansion loop was faulty and caused an
infinite loop when no change occured (i.e. no variables were expanded)
2018-04-25 09:35:42 +02:00
Dan Čermák
740e4ecad1
[testsuite] Improve output in case of errors
...
use the msg= option of unittest.assertXyz() to improve the error
reporting
2018-04-23 10:47:52 +02:00
Dan Čermák
b6d93558ee
[testsuite] Added option to use different encodings for the output
2018-04-23 10:23:50 +02:00
Dan Čermák
dd3bcaf41f
[testsuite] Added check that test list have same length
...
commands, retval, stdout & stderr are lists
if they don't have the same length, they are truncated to the smallest list
thereby silently dropping test cases
=> now a ValueError is raised preventing that
2018-04-23 10:17:36 +02:00
Dan Čermák
11cd1eabf8
[testsuite] Addapt tests to new test suite structure
2018-04-23 10:17:36 +02:00
Dan Čermák
c40c90141f
[testsuite] Add debug mode & search directory to runner
2018-04-23 10:17:36 +02:00
Dan Čermák
4656af73bc
[testsuite] Fix variable names in suite config
...
The variables contained a minus which must not be used in template string
variables.
=> Use underscores instead
2018-04-23 10:17:36 +02:00
Dan Čermák
cac0bc9656
[testsuite] Adapt documentation to recent changes
...
- change from {var} => $var
- use metaclass instead of direct inheritance
2018-04-23 10:17:17 +02:00
Dan Čermák
bd9d085141
[testsuite] Refactor test suite to use metaclasses & template module
...
The testsuite now uses python's template module for string substitutions which
allows for a more natural substitution syntax known from the shell. Also, it
allows to run the substitutions multiple times, which is not possible with
string.format().
The heavy-lifting is now performed via a metaclass, which expands all variables
on the class creation.
2018-04-21 00:59:59 +02:00
Luis Diaz Mas
ffab622633
Add reproducer for #247
2018-04-02 17:46:29 +02:00
Luis Diaz Mas
08aa27ae37
Add reproducers for #253 and #246
2018-04-01 09:21:20 +02:00
Dan Čermák
1f32207271
[testsuite] rename test_issue_208 -> test_CVE_2017_17722
...
the issue got a CVE assigned
2018-03-28 10:07:41 +02:00
Dan Čermák
4533d60e74
[testsuite] Added helper function for system_tests.Case.compare_stderr
2018-03-28 10:07:41 +02:00
Dan Čermák
02f60f11ae
[testsuite] Reimplemented a part of system_tests.CopyFiles
...
- main file handling functionality is performed by a base class => can
be easier extended if required
2018-03-28 10:07:41 +02:00
Dan Čermák
3b01d5205a
[testsuite] Port all regression tests for github issues
2018-03-28 10:07:41 +02:00
Dan Čermák
3cbb031442
[testsuite] Unified CVE test cases
2018-03-28 10:07:41 +02:00
Dan Čermák
e597e5b811
[testsuite] Fixed CVE number
2018-03-28 10:07:41 +02:00
Dan Čermák
c754ca6d14
[testsuite] Corrected class name, issue had no CVE assigned
2018-03-28 10:07:41 +02:00
Dan Čermák
71f9a24027
[testsuite] Unified variable names
2018-03-28 10:07:32 +02:00
Dan Čermák
48982697b0
Add reproducer for CVE-2017-1000126 to the test suite
2018-03-27 23:46:13 +02:00
Dan Čermák
59b148aee9
Add regression test for CVE-2017-18005
2018-03-20 21:30:06 +01:00
Dan Čermák
cabf756553
Add reproducer for #170 to the test suite
2018-02-22 16:42:51 +01:00
Dan Čermák
73ac02d7f2
Added test for Sigma lenses
2018-02-21 12:57:45 +01:00
Dan Čermák
3965a44df2
Replace error variable names in test suite with enum error codes
2018-02-18 10:44:05 +01:00
Dan Čermák
a15f090f81
Modified test suite so that case sensitive keys are possible
2018-02-18 10:42:16 +01:00