-
Notifications
You must be signed in to change notification settings - Fork 5
/
ToDo
177 lines (131 loc) · 5.07 KB
/
ToDo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
Note: ToDo entries with "*" are done, "-" are really ToDo
- add a check, if a TC ends if prompt is read.
- [simplify] create a "setup" script, which simplify setup after
tbot is installed.
- [simplify] create testcases, specific for a specific SoC, so board
testcases can use them. For Example, create a tc_soc_imx6_xxx.py
testcases, which contains testcases we want to run on an imx based
board.
- [simplify] make it possible to move virtual lab specific config options
into lab specific config files, and include them in board
config files
- [simplify] make a documentation of "tbot conventions"
currently this are my personal conventions, if there are more
user try to identify my conventions and discuss and document them
currently they are hidden in default settings of tbot variables ...
- [simplify] Get Vars for TC from U-Boot Code
started with, see TC:
src/tc/tc_workfd_get_uboot_config_hex.py
src/tc/tc_workfd_get_uboot_config_string.py
- simplify tbot usage ... all ToDo points marked with [simplify]
goal: show tbot installation/usage on 2 pages ... (maybe 1 is possible ?)
- one for tbot installation and how to setup a virtual lab
- one for how to setup a board testcase
see some ToDo point here ...
- introduce a connection class and use pexpect
get rid of paramiko
* instead printing fd (0 or 1) use the connection
name.
- automatically create a documentation for all testcases
default_vars and tbotlib functions.
- use sphinx for this task, see doc_sphinx
* testcase
link to other testcases, defaultvars (may show them if
mouse is over the name)
* defaultvars
- tbotlib
* add manpages for the testcases / variables
* create a pdf
* use a dict() for all the parameters
instead using src/common/tc_defaults.py
- cleanup tbot code
- get rid of all retry places
- clean up tbotlib function names
- introduce a "layering" like yocto do, so U-Boot TC can integrated
into U-Boot source code.
Proposal:
introduce subdirs in "src/tc"
lab: all lab specific stuff
lab/common: common lab stuff (for example ssh handling)
lab/denx: denx specific
u-boot: all u-boot tests
u-boot/common: common u-boot tc
u-boot/duts: DUTS tc
u-boot-dxr2: all u-boot dxr2 board specific tc
board: board tc
board/common: common boar tc
board/dxr2: all tc for dxr2 board
linux: all linux tc
linux/common: common linux tc
linux/dxr2
tbot: common tbot tc
tbot/
- need a mechanism in tbot, how it gets automatically for example
U-Boot TC from U-Boot source...
* search TC names in src/tc through all subdirs
- simplify tbot log output (seperate a lot of output which is currently
in INFO logging level, to another logging level)
- make the timestamp configurable
- call testcases with arguments
- now possible, but have to convert all testcases
- if tbot fails, start a git biscect section
and find out, which commit did it
done!
src/tc/tc_board_git_bisect.py
- store commit id when tc is good -> last good commit
id is known
src/tc/tc_board_tqm5200s_ub_comp_install.py
saves the u-boot.bin, if u-boot boots
-> if u-boot boot fails with new u-boot
old u-boot is known -> now extract from last
working the commit id ... and start:
src/tc/tc_board_git_bisect.py
then this is done!
- testcases for yocto
- install yocto
- install meta ..
- patch bblayers.conf
- patch local.conf
- call bitbake
- cp images
- gpio tests
- check registers with devmem
- use readreg
- Delete from the tc_workfd_apply_patchwork_patches_list_hand
python list an entry, if while "git am" the patch "git am"
detects, that this patch is already in mainline.
- WDT timeout runtime changeable
- statistic/report module
TC report start end (automatically) events to it
TC specific events possible
automatic timestamp generate
also log events from each read/write
-> at the end a specific format of the result can be
generated, thorugh modules which are registered
as backends on the event module.
for eample:
- cdash (notification to a central server, push principle)
- dot graph (generate a graph from tcs for visual representation)
- xunit report (xml file for Jenkins visualization)
- DULG: create DULG specific files
- kernelCI
- format log output into a readable format
may something like output from u-boot:/tools/test/py
- end of line detection
As the end of a line is \n, and tbot checks end of line with
\n this maybe leads into errors, if the COLUMNS of the linux
terminal is not long enough, and insert a \n ...
`echo $COLUMNS` must show a value, which can hold the longest
command tbot send ...
I currently set:
```
stty cols 155
export TERM=vt200
```
which I think is not the best way to do it ...
Suggestions welcome
- rename functionnames
rename functionnames in tbotlib to more meaningful names
Oh, yes ... and maybe extract more tbotlib functions
into testcases.
- rework tbot completly into a more pythonic style