From: oharboe Date: Thu, 24 Apr 2008 09:26:50 +0000 (+0000) Subject: Edgar's new test cases. X-Git-Tag: v0.1.0~675 X-Git-Url: https://review.openocd.org/gitweb?p=openocd.git;a=commitdiff_plain;h=6ef27f4534b7a8926866b1156b6a309fc7c3d300;ds=sidebyside Edgar's new test cases. git-svn-id: svn://svn.berlios.de/openocd/trunk@614 b42882b7-edfa-0310-969c-e2dbd0fdcd60 --- diff --git a/testing/index.html b/testing/index.html index b8a8b21945..57085b9523 100644 --- a/testing/index.html +++ b/testing/index.html @@ -8,10 +8,11 @@

Note that this testing document does not have anything to do with testing that is done before committing to svn. It is a test document for released code. Pre-commit testing - is done mostly by the developer who has written the change. Release testing is + is done mostly by the developer who has written the change. Sometimes code is committed + to synchronize work, even if it has known problems. Release testing is done on code believed to be stable, often a couple of weeks old, and not by the developers, but rather users and community testers who has the requisite hardware - and test setup. Also the testing can take place over an extended period of time. + and test setup. Also the testing will take place over an extended period of time.

All of the above makes it imperative that there can be no doubt about *which* code is tested and thus all tests refer to committed code by subversion number. @@ -19,318 +20,18 @@ OpenOCD trunk is work in progress. Expect it to change daily and to have some quirks.

If you need the latest released and tested version, look for binary snapshots of OpenOCD. Worst case look up the test result table below for the features that are important to you and extract and build the version that has the right cocktail of working features for you. You can also work with the community to address the problems you are seing. Testing work and bug reports are highly appreciated.

The OpenOCD community may decide to create release branches. If this happens, then a branch will be created from OpenOCD trunk. The particular version to create that branch might be an older version rather than the latest and greatest. Fixes are then ported to that release branch from OpenOCD trunk.

Vocabulary

- - - - - - - - - - - - - -

Passed version	The latest branch and version on which the test is known to pass
Broken version	The latest branch and version on which the test is known to fail. n/a when older than passed version.
ID	A unqiue ID to refer to a test. The unique numbers are maintained in this file. Note that the same test can be run on different hardware/interface. Each combination yields a unique id.

OpenOCD test results

- These tests can be performed on any JTAG device as long as they are executed using the unmodified code from SVN. -

The latest version in which the test is known to have passed is in the table below.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Unique ID	Synopsis	JTAG device	Passed version	Broken version
fill_malloc	Fill malloc() memory with garbage	n/a	n/a	n/a
ocd1	Telnet Windows	n/a	n/a	n/a
ocd2	Telnet Linux	n/a	n/a	n/a
ocd3	Telnet Cygwin	n/a	n/a	n/a
ocd4	ARM7 debugging	n/a	n/a	n/a
SAM9260	SAM9260 debugging	ft2232	500	n/a
xscale1	XScale debugging	bitbang	505	n/a
xscale2	XScale debugging	FT2232	202	n/a
bdte-ram1	str710 ram debugging	JTAGkey	592	n/a
bdte-rom2	str710 rom debugging	JTAGkey	592	n/a
bdte-ram3	str912 ram debugging	JTAGkey	592	n/a
bdte-rom4	str912 rom debugging	JTAGkey	592	n/a
bdte-ram5	lpc2148 ram debugging	JTAGkey	592	n/a
bdte-rom6	lpc2148 rom debugging	JTAGkey	592	n/a
bdte-ram7	lpc2294 ram debugging	JTAGkey	592	n/a
bdte-rom8	lpc2294 rom debugging	JTAGkey	592	n/a
bdte-ram9	sam7s256 ram debugging	JTAGkey	592	n/a
bdte-rom10	sam7s256 rom debugging	JTAGkey	592	n/a
bdte-ram11	sam7x256 ram debugging	JTAGkey	592	n/a
bdte-rom12	sam7x256 rom debugging	JTAGkey	592	n/a
bdte-ram13	at91r40008 ram debugging	JTAGkey	592	n/a

OpenOCD JTAG device test results

- Each JTAG device must be tested - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

ID	Synopsis	Passed version	Broken version
jtag1	Parport	n/a	n/a
jtag2	JTAGkey	592	n/a
jtag3	Turtelizer2	592	n/a
jtag4	JTAGkey	536	n/a
jtag5	add new one	n/a	n/a

jtag1:

jtag2: Tested on Windows XP Prof. (SP2) with original FTDI driver.

jtag3: Tested on Windows XP Prof. (SP2) with original FTDI driver.

jtag4: Tested on Mac OS X (10.5.2, Intel) with libftdi-0.10 and libusb-0.1.12

jtag5:

OpenOCD JTAG device speed test result

The test result is in KB/sec.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

ID	Synopsis	r320	r420	r423	r459	r517	r536	r592
speed1	JTAGkey	93	64	93	93	93	93	98
speed2	JTAGkey	n/a	n/a	n/a	n/a	52	52	n/a
speed3	add new one	n/a	n/a	n/a	n/a	n/a	n/a	n/a

Policy on removing features from OpenOCD

- If a feature in OpenOCD is known to be broken and nobody has submitted a fix and the feature is causing trouble for maintainence, it can be removed from OpenOCD trunk. The threshold for temporarily removing something from OpenOCD trunk is low to ease maintainence and place the burden of maintainence on those that care about a feature. -

Note that code is never deleted from OpenOCD svn, it remains in svn so if somebody sees a feature removed that they would like kept, they have but to port and fix that feature back up to main trunk. This document can be helpful in this regard in that the latest working version and the known broken version may be listed.

Policy on adding features from OpenOCD

- To add a feature to OpenOCD, generally it should not break any existing features and it should be functional and the code reasonably readable and useful to others in the OpenOCD community. The code does not have to be completed. Work in progress is fine for OpenOCD trunk. -

Also new tests should be defined. Note that the code does not have to pass all the tests. In fact it can be helpful to have tests to describe facets that really should be working, but aren't done yet.

ocd4 - ARM7 debugging

- Connect to ARM7 device(any), use GDB load to load a program into RAM and single halt, resume and single step. -

bdte-ram (Basic debugging test with Eclipse in RAM)

This test was made under Eclipse with the Zylin Embedded CDT plugin. For the GDB "Initialize commands" take a look in the examples/<target>/prj/eclipse_ram.gdb file.

Start debugging, the debugger should stop at main. set some breakpoints and "Resume". If the debugger hit a breakpoint check if the "Variables" looks correct. Remove some breakpoints and "Resume" again. If the target is running, use the "Suspend" function and use "Step Into" or "Step Over" through the source. Even open the "Disassembly" view and enable the "Instruction Stepping Mode". Now you can single step through the assembler source. Use "Resume" again to run the program, set a breakpoint while the target is running. Check if you can inspect the variables with the mouse over. Play a little with the target...

bdte-rom (Basic debugging test with Eclipse in ROM)

This test was made under Eclipse with the Zylin Embedded CDT plugin. For the GDB "Initialize commands" take a look in the examples/<target>/prj/eclipse_rom.gdb file.

Start debugging, the debugger should download and store the program in the flash of the target.

Now you can make some tests like described in the bdte-ram section above too.

speed1 - Download speed test

For this test a STR710 with external memory was used. The example project can be found under examples/STR710JtagSpeed. Here Eclipse or the arm-elf-gdb can be used to download the test.elf file into the RAM. The result of the GDB can look like:

Loading section .text, size 0x6019c lma 0x62000000
- Start address 0x62000040, load size 393628
- Transfer rate: 93 KB/sec, 2008 bytes/write.

In this example a speed of 93 KB/sec was reached. The hardware which was used for the test can be found here.

The test was made on Windows XP Prof. (SP2) with a JTAGkey and the original FTDI driver.

speed2 - Download speed test

Loading section .text, size 0x6019c lma 0x62000000
- Start address 0x62000040, load size 393628
Transfer rate: 52 KB/sec, 2008 bytes/write.

In this example a speed of 52 KB/sec was reached. The hardware which was used for the test can be found here.

The test was made on Mac OS X (10.5.2, Intel) with a JTAGkey and the following driver:

- libftdi 0.10
- - libusb 0.1.12

OpenOCD smoketests

+ This is a set of tests that exercise the entire OpenOCD system and various targets. It + is a small suite of systemwide smoketests. +

+ Smoketests +

Test cases

+ Additionally OpenOCD has test cases that target specific functionality more precisely. +

+ A full release test must include both smoketests and unit testing. +

+ Test cases \ No newline at end of file diff --git a/testing/results/template.html b/testing/results/template.html new file mode 100644 index 0000000000..906925870c --- /dev/null +++ b/testing/results/template.html @@ -0,0 +1,18 @@ + + + + + + Testcases + + + + + + + + +
Test Interface Target Result
CON001 FILL IN! FILL IN! PASS/FAIL
CON002 FILL IN! FILL IN! PASS/FAIL
RES001 FILL IN! FILL IN! PASS/FAIL
RES002 FILL IN! FILL IN! PASS/FAIL
RES003 FILL IN! FILL IN! PASS/FAIL
DBG001 FILL IN! FILL IN! PASS/FAIL
+ + + \ No newline at end of file diff --git a/testing/smoketests.html b/testing/smoketests.html new file mode 100644 index 0000000000..cc8a553e75 --- /dev/null +++ b/testing/smoketests.html @@ -0,0 +1,315 @@ + + + + + +

OpenOCD smoketest results

+ These tests can be performed on any JTAG device as long as they are executed using the unmodified code from SVN. +

The latest version in which the test is known to have passed is in the table below.

Vocabulary

+ + + + + + + + + + + + + +

Passed version	The latest branch and version on which the test is known to pass
Broken version	The latest branch and version on which the test is known to fail. n/a when older than passed version.
ID	A unqiue ID to refer to a test. The unique numbers are maintained in this file. Note that the same test can be run on different hardware/interface. Each combination yields a unique id.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Unique ID	Synopsis	JTAG device	Passed version	Broken version
fill_malloc	Fill malloc() memory with garbage	n/a	n/a	n/a
ocd1	Telnet Windows	n/a	n/a	n/a
ocd2	Telnet Linux	n/a	n/a	n/a
ocd3	Telnet Cygwin	n/a	n/a	n/a
ocd4	ARM7 debugging	n/a	n/a	n/a
SAM9260	SAM9260 debugging	ft2232	500	n/a
xscale1	XScale debugging	bitbang	505	n/a
xscale2	XScale debugging	FT2232	202	n/a
bdte-ram1	str710 ram debugging	JTAGkey	536	n/a
bdte-rom2	str710 rom debugging	JTAGkey	536	n/a
bdte-ram3	str912 ram debugging	JTAGkey	536	n/a
bdte-rom4	str912 rom debugging	JTAGkey	536	n/a
bdte-ram5	lpc2148 ram debugging	JTAGkey	536	n/a
bdte-rom6	lpc2148 rom debugging	JTAGkey	536	n/a
bdte-ram7	lpc2294 ram debugging	JTAGkey	536	n/a
bdte-rom8	lpc2294 rom debugging	JTAGkey	536	n/a
bdte-ram9	sam7s256 ram debugging	JTAGkey	536	n/a
bdte-rom10	sam7s256 rom debugging	JTAGkey	536	n/a
bdte-ram11	sam7x256 ram debugging	JTAGkey	517	n/a
bdte-rom12	sam7x256 rom debugging	JTAGkey	517	n/a
bdte-ram13	at91r40008 ram debugging	JTAGkey	536	n/a

OpenOCD JTAG device test results

+ Each JTAG device must be tested + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

ID	Synopsis	Passed version	Broken version
jtag1	Parport	n/a	n/a
jtag2	JTAGkey	536	n/a
jtag3	Turtelizer2	536	n/a
jtag4	JTAGkey	536	n/a
jtag5	add new one	n/a	n/a

jtag1:

jtag2: Tested on Windows XP Prof. (SP2) with original FTDI driver.

jtag3: Tested on Windows XP Prof. (SP2) with original FTDI driver.

jtag4: Tested on Mac OS X (10.5.2, Intel) with libftdi-0.10 and libusb-0.1.12

jtag5:

OpenOCD JTAG device speed test result

The test result is in KB/sec.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

ID	Synopsis	r320	r420	r423	r459	r517	r536
speed1	JTAGkey	93	64	93	93	93	93
speed2	JTAGkey	n/a	n/a	n/a	n/a	52	52
speed3	add new one	n/a	n/a	n/a	n/a	n/a	n/a

Policy on removing features from OpenOCD

+ If a feature in OpenOCD is known to be broken and nobody has submitted a fix and the feature is causing trouble for maintainence, it can be removed from OpenOCD trunk. The threshold for temporarily removing something from OpenOCD trunk is low to ease maintainence and place the burden of maintainence on those that care about a feature. +

Policy on adding features from OpenOCD

+ To add a feature to OpenOCD, generally it should not break any existing features and it should be functional and the code reasonably readable and useful to others in the OpenOCD community. The code does not have to be completed. Work in progress is fine for OpenOCD trunk. +

ocd4 - ARM7 debugging

+ Connect to ARM7 device(any), use GDB load to load a program into RAM and single halt, resume and single step. +

bdte-ram (Basic debugging test with Eclipse in RAM)

This test was made under Eclipse with the Zylin Embedded CDT plugin. For the GDB "Initialize commands" take a look in the examples/<target>/prj/eclipse_ram.gdb file.

bdte-rom (Basic debugging test with Eclipse in ROM)

This test was made under Eclipse with the Zylin Embedded CDT plugin. For the GDB "Initialize commands" take a look in the examples/<target>/prj/eclipse_rom.gdb file.

Start debugging, the debugger should download and store the program in the flash of the target.

Now you can make some tests like described in the bdte-ram section above too.

speed1 - Download speed test

Loading section .text, size 0x6019c lma 0x62000000
+ Start address 0x62000040, load size 393628
+ Transfer rate: 93 KB/sec, 2008 bytes/write.

In this example a speed of 93 KB/sec was reached. The hardware which was used for the test can be found here.

The test was made on Windows XP Prof. (SP2) with a JTAGkey and the original FTDI driver.

speed2 - Download speed test

Loading section .text, size 0x6019c lma 0x62000000
+ Start address 0x62000040, load size 393628
Transfer rate: 52 KB/sec, 2008 bytes/write.

In this example a speed of 52 KB/sec was reached. The hardware which was used for the test can be found here.

The test was made on Mac OS X (10.5.2, Intel) with a JTAGkey and the following driver:

- libftdi 0.10
+ - libusb 0.1.12

+ + + \ No newline at end of file

Test	Interface	Target	Result
CON001	FILL IN!	FILL IN!	PASS/FAIL
CON002	FILL IN!	FILL IN!	PASS/FAIL
RES001	FILL IN!	FILL IN!	PASS/FAIL
RES002	FILL IN!	FILL IN!	PASS/FAIL
RES003	FILL IN!	FILL IN!	PASS/FAIL
DBG001	FILL IN!	FILL IN!	PASS/FAIL