Commit Graph

68 Commits

Author SHA1 Message Date
Dan Čermák
dadd1d19f9 Add reproducer for #400 to the test suite 2018-08-21 13:46:53 +02:00
Luis Diaz Mas
3d35b93fc0 Skip test 1054 unless timezone is UTC 2018-07-31 18:18:14 +02:00
Luis Diaz Mas
145f8fc456 Port redmine issues from the bash test suite to the python one.
* Port redmine issue 1024
 * Port redmine issue 1026
 * Port redmine issue 1040
 * Port redmine issue 1044
 * Port redmine issue 1053
 * Port redmine issue 1054 (not finished yet; I found some issues there)
 * Port redmine issue 1058
 * Port redmine issue 1062
 * Port redmine issue 1080
 * Port redmine issue 1108
 * Port redmine issue 1112
 * Port redmine issue 1114
 * Port redmine issue 1122
 * Port redmine issue 1140
 * Port redmine issue 1144
 * Port redmine issue 1145
 * Port redmine issue 1153
 * Port redmine issue 1155
 * Port redmine issue 1166
 * Port redmine issue 1167
 * Port redmine issue 1170
 * Replace escaped chars in 1054
 * Add brackets in exiv2json::main()
 * Link all sample apps against exiv2lib
 * Changes in 1054
 * test 1054: fix dates depending on Local time
 * Do not run make with VERBOSE=1
 * Use system_tests.path
 * Fix windows issues with quotes
 * Use system_tests.path
 * Use itertools to simplify test code
2018-07-31 18:18:14 +02:00
Dan Čermák
e67910a669 [tests] Change name of test for #365 and #366
Issues got a CVE assigned
2018-07-30 00:32:07 +02:00
Dan Čermák
f522cbf460 [testsuite] Add reproducer for #378/CVE-2018-14046 to the testsuite 2018-07-16 23:40:23 +02:00
Dan Čermák
b517f2e13b Add regression test for #366 to the testsuite 2018-06-11 22:55:49 +02:00
Dan Čermák
9b08354aca Add reproducer for #365 to the testsuite 2018-06-11 22:55:49 +02:00
Dan Čermák
653814afb0 [testsuite] Allow setting the binary location for make tests 2018-06-07 10:19:59 +02:00
Dan Čermák
35e7037f86 [testsuite] Change the calling order of setUp & tearDown in decorator
The FileDecoratorBase injects a new setUp & tearDown function. These new
functions would call the old setUp & tearDown in an inconvenient order: e.g. the
child class CopyFiles would at first call the user provided setUp and then copy
the files. This makes it impossible to perform some action on the file copy in
setUp.
=> This commit changes the call order, so that setUp & tearDown always "see" the
finished environment after file copies are in place and before any cleanup took
place.
2018-06-06 13:59:45 +02:00
Dan Čermák
39b7151b2e [testsuite] Port redmine issues 831 to 937 to the new testsuite 2018-06-06 13:59:34 +02:00
Dan Čermák
7e1cd7d1b6 [testsuite] Fix name of regression test #283
issue got a CVE assigned
2018-06-06 13:31:53 +02:00
Luis Diaz Mas
5bd26ee82b Port redmine issues 1179-1305
- Authorship: Luis Díaz Más & Dan Čermák
2018-06-01 07:56:13 +02:00
Dan Čermák
d7f4453dbd [testsuite] Remove usage of $cat via hooks 2018-05-29 10:21:13 +02:00
Dan Čermák
41ac25c4f4 [testsuite] Add hooks & documentation for hooks 2018-05-29 10:21:13 +02:00
Dan Čermák
23e29e9527 [testsuite] Use system_tests.path() in some test cases 2018-05-29 10:21:13 +02:00
Dan Čermák
4e32929956 [testsuite] Add notes concerning escaping in commands 2018-05-29 10:21:13 +02:00
Dan Čermák
53c265f8a7 [testsuite] Add path conversion function 2018-05-29 10:21:13 +02:00
Dan Čermák
300b1dc0ef [testsuite] Testsuite injects config file values into its namespace 2018-05-29 10:21:13 +02:00
Luis Diaz Mas
8eca055058 Add POC + regression test.
Note that we override here the method check_no_ASAN_UBSAN_errors to also
analyze the last two lines of got_stderr.
2018-05-26 09:59:37 +02:00
Luis Díaz Más
b8b94dc67a Escape characters properly once we started using python literals 2018-05-24 11:06:18 +02:00
Luis Díaz Más
48078e9c8b Fix tests on windows by using os.path.join 2018-05-24 11:06:18 +02:00
Luis Díaz Más
72faa7331d Fix issues with python strings & literals 2018-05-24 11:06:18 +02:00
Dan Čermák
90c8d754b7 [testsuite] Call subprocess.Popen() with shell=True on Windows 2018-05-24 11:06:18 +02:00
Dan Čermák
939ded283f [testsuite] check stderr before stdout
That way you get to see error messages about wrong filenames instead of an
empty stdout
2018-05-24 11:06:18 +02:00
Dan Čermák
f55cdce761 [testsuite] Port redmine issues 498-825 2018-05-24 11:06:18 +02:00
Dan Čermák
2356984274 [testsuite] Ported redmine bugfixes 426-495 2018-05-24 11:06:18 +02:00
Luis Diaz Mas
2fb00c8a16 Analyze minimum needed number of null separators in PngChunk::parseTXTChunk
This commit fixes the heap-buffer-overflow in PngChunk::parseTXTChunk.

According to the specification:
http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html

There must be 2 null separators when we start to analyze the language tag.
2018-05-23 10:57:13 +02:00
Luis Diaz Mas
234e5794f6 Add tests for #306 2018-05-23 10:57:13 +02:00
Daniel Zucchetto
cb5135f772 Add test for Canon M100 ModelID 2018-05-21 23:35:39 +02:00
Robin Mills
4239ff6734 Test for Tamron SP AF 150-600mm F/5-6.3 VC USD G2 (A022) for Nikon 2018-05-06 12:00:20 +01:00
Dan Čermák
b364e075c6 [testsuite] Implement DeleteFiles decorator
- Refactor FileDecoratorBase to use an arbitrary property in the decorated
  class where the list of files is saved
- Add DeleteFiles decorator which deletes all files passed to it after the test
  finished
2018-04-25 09:41:50 +02:00
Dan Čermák
d1714f6564 [testsuite] Fix typo and obsolete syntax 2018-04-25 09:35:42 +02:00
Dan Čermák
fe98936375 [testsuite] Fixed infinite iteration in CaseMeta.__new__
The loop condition in the variable expansion loop was faulty and caused an
infinite loop when no change occured (i.e. no variables were expanded)
2018-04-25 09:35:42 +02:00
Dan Čermák
740e4ecad1 [testsuite] Improve output in case of errors
use the msg= option of unittest.assertXyz() to improve the error
reporting
2018-04-23 10:47:52 +02:00
Dan Čermák
b6d93558ee [testsuite] Added option to use different encodings for the output 2018-04-23 10:23:50 +02:00
Dan Čermák
dd3bcaf41f [testsuite] Added check that test list have same length
commands, retval, stdout & stderr are lists
if they don't have the same length, they are truncated to the smallest list
thereby silently dropping test cases
=> now a ValueError is raised preventing that
2018-04-23 10:17:36 +02:00
Dan Čermák
11cd1eabf8 [testsuite] Addapt tests to new test suite structure 2018-04-23 10:17:36 +02:00
Dan Čermák
c40c90141f [testsuite] Add debug mode & search directory to runner 2018-04-23 10:17:36 +02:00
Dan Čermák
4656af73bc [testsuite] Fix variable names in suite config
The variables contained a minus which must not be used in template string
variables.
=> Use underscores instead
2018-04-23 10:17:36 +02:00
Dan Čermák
cac0bc9656 [testsuite] Adapt documentation to recent changes
- change from {var} => $var
- use metaclass instead of direct inheritance
2018-04-23 10:17:17 +02:00
Dan Čermák
bd9d085141 [testsuite] Refactor test suite to use metaclasses & template module
The testsuite now uses python's template module for string substitutions which
allows for a more natural substitution syntax known from the shell. Also, it
allows to run the substitutions multiple times, which is not possible with
string.format().
The heavy-lifting is now performed via a metaclass, which expands all variables
on the class creation.
2018-04-21 00:59:59 +02:00
Luis Diaz Mas
ffab622633 Add reproducer for #247 2018-04-02 17:46:29 +02:00
Luis Diaz Mas
08aa27ae37 Add reproducers for #253 and #246 2018-04-01 09:21:20 +02:00
Dan Čermák
1f32207271 [testsuite] rename test_issue_208 -> test_CVE_2017_17722
the issue got a CVE assigned
2018-03-28 10:07:41 +02:00
Dan Čermák
4533d60e74 [testsuite] Added helper function for system_tests.Case.compare_stderr 2018-03-28 10:07:41 +02:00
Dan Čermák
02f60f11ae [testsuite] Reimplemented a part of system_tests.CopyFiles
- main file handling functionality is performed by a base class => can
  be easier extended if required
2018-03-28 10:07:41 +02:00
Dan Čermák
3b01d5205a [testsuite] Port all regression tests for github issues 2018-03-28 10:07:41 +02:00
Dan Čermák
3cbb031442 [testsuite] Unified CVE test cases 2018-03-28 10:07:41 +02:00
Dan Čermák
e597e5b811 [testsuite] Fixed CVE number 2018-03-28 10:07:41 +02:00
Dan Čermák
c754ca6d14 [testsuite] Corrected class name, issue had no CVE assigned 2018-03-28 10:07:41 +02:00