Commit Graph

402 Commits

Author SHA1 Message Date
Dan Čermák
7bd929de31
[tests] Add regression test for #756 2019-03-25 23:56:57 +01:00
Dan Čermák
a557c7f477
[tests] Add regression test for second bug from #590 2019-02-26 00:09:21 +01:00
Luis Diaz Mas
fce84fdfdf Adapt test after updating nikon lens database 2018-12-23 20:18:44 +01:00
Luis Díaz Más
dc04cc92eb Add test reproducer for #561 2018-11-26 16:39:00 +01:00
Luis Diaz Mas
9a84d4b64f Skip python tests using -pR 2018-11-22 18:19:08 +01:00
Luis Díaz Más
e84b812d8a Tests: increase timeout 2018-11-11 12:27:26 +01:00
Luis Díaz Más
2ea9039879 Adapt fujifilm regression tests 2018-11-07 12:56:23 +01:00
Luis Diaz Mas
b07c480a0a Add regression test for #511 2018-11-06 14:40:36 +01:00
Luis Diaz Mas
b6bc4262c4 Add regression test for #426 2018-11-06 13:17:35 +01:00
Luis Díaz Más
274b65a671 Add reproducer for #460 2018-10-14 11:58:25 +02:00
Luis Díaz Más
3f97960a1c Add reproducer for #457 2018-10-13 11:37:57 +02:00
Dan Čermák
f6d775b400 [testsuite] Add reproducers for #263
This issue got resolved by #180 and #461.
2018-10-12 00:41:46 +02:00
Dan Čermák
d224f897b0 [testsuite] Add reproducer for #216
The bug got resolved by PR #461 (slices).
2018-10-12 00:41:46 +02:00
Dan Čermák
eeb520bf39 [testsuite] Add reproducer for second bug from #159
The bug described in the issue got resolved by PR #461 (slices).
2018-10-12 00:41:46 +02:00
Dan Čermák
1af8e2875e [testsuite] Add reproducers for CVE-2017-17724 & #209, #211 2018-10-11 14:06:16 +02:00
Roberto C. Sánchez
c03f73268f Prevent SIGABRT on excessive subBox length in jp2image.cpp
This fixes CVE-2018-9145
2018-10-10 11:52:39 +02:00
Dan Čermák
19bb57ff25 Add reproducer for #262 to the test suite 2018-09-13 11:18:29 +02:00
Dan Čermák
03799c75e1 [testsuite] Add support for binary output to check_no_ASAN_UBSAN_errors 2018-09-13 11:18:29 +02:00
Dan Čermák
75415693d8 [testsuite] Update testsuite after printStructure deletion 2018-09-13 11:18:29 +02:00
Luis Díaz Más
7d32da890b Add remaining pocs for the issue 428 and adapt the regression test 2018-09-11 09:49:59 +02:00
Luis Díaz Más
f5b40f3e82 Fix more issues in PngChunk::readRawProfile 2018-09-11 09:49:59 +02:00
Luis Díaz Más
a6a1c31140 Add integration test to reproduce the issue 2018-09-11 09:49:59 +02:00
Robin Mills
74bc0e0535 Changes for cross-platform getopt 2018-09-01 20:19:45 +02:00
Dan Čermák
0f43d0f49d [Travis CI] Add new matrix entry: run test suite under Valgrind 2018-08-30 16:26:56 +02:00
Dan Čermák
ed28e15152 [testsuite] Add option to run all commands under valgrind 2018-08-30 16:26:56 +02:00
Dan Čermák
c7a3a57aff [testsuite] Port redmine issue #1137 (requires stdin passing) 2018-08-30 16:26:56 +02:00
Dan Čermák
992877bb39 [testsuite] Implement the supplying of stdin to commands 2018-08-30 16:26:56 +02:00
Dan Čermák
872e158173 [testsuite] Port redmine issue 1043
This test used shell wildcards, which we work around by creating the
list of files to process manually.
2018-08-30 16:26:56 +02:00
Dan Čermák
bf43cbbccc [testsuite] Set env var TZ in redmine issue 1054 & remove skip
The manual export TZ=UTC in run.sh is thereby no longer needed
2018-08-30 16:26:27 +02:00
Dan Čermák
9cfcfb98a2 [testsuite] Add feature to set environment variables 2018-08-30 12:33:31 +02:00
Dan Čermák
116f5fe5bd [testsuite] Port redmine issue 1074 using the binary output feature 2018-08-30 12:33:31 +02:00
Dan Čermák
4f8c150582 [testsuite] Added binary output checking to the test suite 2018-08-30 12:33:16 +02:00
Dan Čermák
bf50b582cb [testsuite] Improve PEP8 compliance 2018-08-27 23:28:11 +02:00
Dan Čermák
deb9c23df8 [testsuite] Add option to run tests from a single file only 2018-08-27 23:28:03 +02:00
Dan Čermák
dadd1d19f9 Add reproducer for #400 to the test suite 2018-08-21 13:46:53 +02:00
Luis Diaz Mas
3d35b93fc0 Skip test 1054 unless timezone is UTC 2018-07-31 18:18:14 +02:00
Luis Diaz Mas
145f8fc456 Port redmine issues from the bash test suite to the python one.
* Port redmine issue 1024
 * Port redmine issue 1026
 * Port redmine issue 1040
 * Port redmine issue 1044
 * Port redmine issue 1053
 * Port redmine issue 1054 (not finished yet; I found some issues there)
 * Port redmine issue 1058
 * Port redmine issue 1062
 * Port redmine issue 1080
 * Port redmine issue 1108
 * Port redmine issue 1112
 * Port redmine issue 1114
 * Port redmine issue 1122
 * Port redmine issue 1140
 * Port redmine issue 1144
 * Port redmine issue 1145
 * Port redmine issue 1153
 * Port redmine issue 1155
 * Port redmine issue 1166
 * Port redmine issue 1167
 * Port redmine issue 1170
 * Replace escaped chars in 1054
 * Add brackets in exiv2json::main()
 * Link all sample apps against exiv2lib
 * Changes in 1054
 * test 1054: fix dates depending on Local time
 * Do not run make with VERBOSE=1
 * Use system_tests.path
 * Fix windows issues with quotes
 * Use system_tests.path
 * Use itertools to simplify test code
2018-07-31 18:18:14 +02:00
Dan Čermák
e67910a669 [tests] Change name of test for #365 and #366
Issues got a CVE assigned
2018-07-30 00:32:07 +02:00
Dan Čermák
f522cbf460 [testsuite] Add reproducer for #378/CVE-2018-14046 to the testsuite 2018-07-16 23:40:23 +02:00
Dan Čermák
b517f2e13b Add regression test for #366 to the testsuite 2018-06-11 22:55:49 +02:00
Dan Čermák
9b08354aca Add reproducer for #365 to the testsuite 2018-06-11 22:55:49 +02:00
Dan Čermák
653814afb0 [testsuite] Allow setting the binary location for make tests 2018-06-07 10:19:59 +02:00
Dan Čermák
35e7037f86 [testsuite] Change the calling order of setUp & tearDown in decorator
The FileDecoratorBase injects a new setUp & tearDown function. These new
functions would call the old setUp & tearDown in an inconvenient order: e.g. the
child class CopyFiles would at first call the user provided setUp and then copy
the files. This makes it impossible to perform some action on the file copy in
setUp.
=> This commit changes the call order, so that setUp & tearDown always "see" the
finished environment after file copies are in place and before any cleanup took
place.
2018-06-06 13:59:45 +02:00
Dan Čermák
39b7151b2e [testsuite] Port redmine issues 831 to 937 to the new testsuite 2018-06-06 13:59:34 +02:00
Dan Čermák
7e1cd7d1b6 [testsuite] Fix name of regression test #283
issue got a CVE assigned
2018-06-06 13:31:53 +02:00
Luis Diaz Mas
5bd26ee82b Port redmine issues 1179-1305
- Authorship: Luis Díaz Más & Dan Čermák
2018-06-01 07:56:13 +02:00
Dan Čermák
d7f4453dbd [testsuite] Remove usage of $cat via hooks 2018-05-29 10:21:13 +02:00
Dan Čermák
41ac25c4f4 [testsuite] Add hooks & documentation for hooks 2018-05-29 10:21:13 +02:00
Dan Čermák
23e29e9527 [testsuite] Use system_tests.path() in some test cases 2018-05-29 10:21:13 +02:00
Dan Čermák
4e32929956 [testsuite] Add notes concerning escaping in commands 2018-05-29 10:21:13 +02:00
Dan Čermák
53c265f8a7 [testsuite] Add path conversion function 2018-05-29 10:21:13 +02:00
Dan Čermák
300b1dc0ef [testsuite] Testsuite injects config file values into its namespace 2018-05-29 10:21:13 +02:00
Luis Diaz Mas
8eca055058 Add POC + regression test.
Note that we override here the method check_no_ASAN_UBSAN_errors to also
analyze the last two lines of got_stderr.
2018-05-26 09:59:37 +02:00
Luis Díaz Más
b8b94dc67a Escape characters properly once we started using python literals 2018-05-24 11:06:18 +02:00
Luis Díaz Más
48078e9c8b Fix tests on windows by using os.path.join 2018-05-24 11:06:18 +02:00
Luis Díaz Más
72faa7331d Fix issues with python strings & literals 2018-05-24 11:06:18 +02:00
Dan Čermák
90c8d754b7 [testsuite] Call subprocess.Popen() with shell=True on Windows 2018-05-24 11:06:18 +02:00
Dan Čermák
939ded283f [testsuite] check stderr before stdout
That way you get to see error messages about wrong filenames instead of an
empty stdout
2018-05-24 11:06:18 +02:00
Dan Čermák
f55cdce761 [testsuite] Port redmine issues 498-825 2018-05-24 11:06:18 +02:00
Dan Čermák
2356984274 [testsuite] Ported redmine bugfixes 426-495 2018-05-24 11:06:18 +02:00
Luis Diaz Mas
2fb00c8a16 Analyze minimum needed number of null separators in PngChunk::parseTXTChunk
This commit fixes the heap-buffer-overflow in PngChunk::parseTXTChunk.

According to the specification:
http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html

There must be 2 null separators when we start to analyze the language tag.
2018-05-23 10:57:13 +02:00
Luis Diaz Mas
234e5794f6 Add tests for #306 2018-05-23 10:57:13 +02:00
Daniel Zucchetto
cb5135f772 Add test for Canon M100 ModelID 2018-05-21 23:35:39 +02:00
Robin Mills
4239ff6734 Test for Tamron SP AF 150-600mm F/5-6.3 VC USD G2 (A022) for Nikon 2018-05-06 12:00:20 +01:00
Dan Čermák
b364e075c6 [testsuite] Implement DeleteFiles decorator
- Refactor FileDecoratorBase to use an arbitrary property in the decorated
  class where the list of files is saved
- Add DeleteFiles decorator which deletes all files passed to it after the test
  finished
2018-04-25 09:41:50 +02:00
Dan Čermák
d1714f6564 [testsuite] Fix typo and obsolete syntax 2018-04-25 09:35:42 +02:00
Dan Čermák
fe98936375 [testsuite] Fixed infinite iteration in CaseMeta.__new__
The loop condition in the variable expansion loop was faulty and caused an
infinite loop when no change occured (i.e. no variables were expanded)
2018-04-25 09:35:42 +02:00
Dan Čermák
740e4ecad1 [testsuite] Improve output in case of errors
use the msg= option of unittest.assertXyz() to improve the error
reporting
2018-04-23 10:47:52 +02:00
Dan Čermák
b6d93558ee [testsuite] Added option to use different encodings for the output 2018-04-23 10:23:50 +02:00
Dan Čermák
dd3bcaf41f [testsuite] Added check that test list have same length
commands, retval, stdout & stderr are lists
if they don't have the same length, they are truncated to the smallest list
thereby silently dropping test cases
=> now a ValueError is raised preventing that
2018-04-23 10:17:36 +02:00
Dan Čermák
11cd1eabf8 [testsuite] Addapt tests to new test suite structure 2018-04-23 10:17:36 +02:00
Dan Čermák
c40c90141f [testsuite] Add debug mode & search directory to runner 2018-04-23 10:17:36 +02:00
Dan Čermák
4656af73bc [testsuite] Fix variable names in suite config
The variables contained a minus which must not be used in template string
variables.
=> Use underscores instead
2018-04-23 10:17:36 +02:00
Dan Čermák
cac0bc9656 [testsuite] Adapt documentation to recent changes
- change from {var} => $var
- use metaclass instead of direct inheritance
2018-04-23 10:17:17 +02:00
Dan Čermák
bd9d085141 [testsuite] Refactor test suite to use metaclasses & template module
The testsuite now uses python's template module for string substitutions which
allows for a more natural substitution syntax known from the shell. Also, it
allows to run the substitutions multiple times, which is not possible with
string.format().
The heavy-lifting is now performed via a metaclass, which expands all variables
on the class creation.
2018-04-21 00:59:59 +02:00
Luis Diaz Mas
ffab622633 Add reproducer for #247 2018-04-02 17:46:29 +02:00
Luis Diaz Mas
08aa27ae37 Add reproducers for #253 and #246 2018-04-01 09:21:20 +02:00
Dan Čermák
1f32207271 [testsuite] rename test_issue_208 -> test_CVE_2017_17722
the issue got a CVE assigned
2018-03-28 10:07:41 +02:00
Dan Čermák
4533d60e74 [testsuite] Added helper function for system_tests.Case.compare_stderr 2018-03-28 10:07:41 +02:00
Dan Čermák
02f60f11ae [testsuite] Reimplemented a part of system_tests.CopyFiles
- main file handling functionality is performed by a base class => can
  be easier extended if required
2018-03-28 10:07:41 +02:00
Dan Čermák
3b01d5205a [testsuite] Port all regression tests for github issues 2018-03-28 10:07:41 +02:00
Dan Čermák
3cbb031442 [testsuite] Unified CVE test cases 2018-03-28 10:07:41 +02:00
Dan Čermák
e597e5b811 [testsuite] Fixed CVE number 2018-03-28 10:07:41 +02:00
Dan Čermák
c754ca6d14 [testsuite] Corrected class name, issue had no CVE assigned 2018-03-28 10:07:41 +02:00
Dan Čermák
71f9a24027 [testsuite] Unified variable names 2018-03-28 10:07:32 +02:00
Dan Čermák
48982697b0 Add reproducer for CVE-2017-1000126 to the test suite 2018-03-27 23:46:13 +02:00
Dan Čermák
59b148aee9 Add regression test for CVE-2017-18005 2018-03-20 21:30:06 +01:00
Dan Čermák
cabf756553 Add reproducer for #170 to the test suite 2018-02-22 16:42:51 +01:00
Dan Čermák
73ac02d7f2 Added test for Sigma lenses 2018-02-21 12:57:45 +01:00
Dan Čermák
3965a44df2 Replace error variable names in test suite with enum error codes 2018-02-18 10:44:05 +01:00
Dan Čermák
a15f090f81 Modified test suite so that case sensitive keys are possible 2018-02-18 10:42:16 +01:00
Dan Čermák
ead7f309b0 Added reproducer for CVE-2017-1000127 2018-02-07 11:16:23 +01:00
Dan Čermák
72de0f96f3 Added reproducer for CVE-2018-4868 to the test suite 2018-02-01 17:55:22 +01:00
Robin Mills
a1f1989cb5
Merge branch 'master' into master 2018-02-01 14:55:07 +00:00
Dan Čermák
80c4d95171 Add regression test for CVE 2017-17669 2018-01-27 14:51:40 +01:00
Dan Čermák
e0ab5aaa87 Add error 14 text as a variable to the test suite 2018-01-27 14:51:40 +01:00
Michał Walenciak
421c6d6723 improvements for issue #208 2018-01-14 08:24:26 +01:00
Michał Walenciak
dba9fba725 test for issue #208 2018-01-13 15:52:50 +01:00
Dan Čermák
9900f523fe Added reproducer for #201 to the test suite 2018-01-06 00:26:09 +01:00
Dan Čermák
5267123186 Ported tiff-test.sh 2018-01-05 00:44:24 +01:00
Dan Čermák
085c9715fc Added some example test cases to the new test suite 2018-01-05 00:43:43 +01:00
Dan Čermák
2e745b0b26 Added replacement for the current regression test suite 2018-01-05 00:42:45 +01:00