Projekt

Obecné

Profil

« Předchozí | Další » 

Revize 9540d301

Přidáno uživatelem Tomáš Ballák před asi 4 roky(ů)

rebase

Zobrazit rozdíly:

.gitignore
9 9
.idea
10 10
dev-dump
11 11
*.lock
12
.vscode-server
13
*__pycache__*
14
*.CSV
15
python-module/venv
12
.vscode-server
modules/crawler/.gitignore
1
*__pycache__*
2
*.CSV
3
python-module/venv
modules/crawler/CrawledData/JIS/ignore.txt
1
ignore.txt
modules/crawler/CrawledData/KOLOBEZKY/ignore.txt
1
ignore.txt
modules/crawler/CrawledData/WIFI/ignore.txt
1
ignore.txt
modules/crawler/DatasetConfigs/JIS.yaml
1
# jmeno datasetu, pod kterym bude zobrazen v aplikaci
2
dataset-name: JIS
3
# root slozka, ktera obsahuje odkazy na dataset
4
url: https://openstore.zcu.cz/
5
# volitelny parameter, ktery specifikuje vzor jmena datasetu, ktera se budou stahovat
6
regex: OD_ZCU_JIS_[0-9][0-9]_[0-9][0-9][0-9][0-9]_CSV.zip
7
# volitelny parametr, ktery udava jednou za kolik dní se budou nová data vyhledávat
8
update-period: 24
9
# pozice jednotlivych zarizeni, ktera jsou v datasetu
10
devices:
11
  - US 005 - z?vora vjezd:
12
      x: UNKNOWN!
13
      y: UNKNOWN!
14

  
15
  - STUD_KL20:
16
      x: UNKNOWN!
17
      y: UNKNOWN!
18

  
19
  - US 005 - m?? vjezd:
20
      x: UNKNOWN!
21
      y: UNKNOWN!
22

  
23
  - Menza4-kasa3:
24
      x: UNKNOWN!
25
      y: UNKNOWN!
26

  
27
  - NTIS-BUFET:
28
      x: UNKNOWN!
29
      y: UNKNOWN!
30

  
31
  - Parkoviste-vjezd:
32
      x: UNKNOWN!
33
      y: UNKNOWN!
34

  
35
  - Menza4-kasa1:
36
      x: UNKNOWN!
37
      y: UNKNOWN!
38

  
39
  - Menza1-kasa-p:
40
      x: UNKNOWN!
41
      y: UNKNOWN!
42

  
43
  - Menza4-kasa2:
44
      x: UNKNOWN!
45
      y: UNKNOWN!
46

  
47
  - L2:
48
      x: UNKNOWN!
49
      y: UNKNOWN!
50

  
51
  - EP-BUFET:
52
      x: UNKNOWN!
53
      y: UNKNOWN!
54

  
55
  - KolaBory-vnitrni:
56
      x: UNKNOWN!
57
      y: UNKNOWN!
58

  
59
  - Menza1-kasa-l:
60
      x: UNKNOWN!
61
      y: UNKNOWN!
62

  
63
  - Zavora-FEL:
64
      x: UNKNOWN!
65
      y: UNKNOWN!
66

  
67
  - Zavora-NTIS-vjezd:
68
      x: UNKNOWN!
69
      y: UNKNOWN!
70

  
71
  - L-Posilovna:
72
      x: UNKNOWN!
73
      y: UNKNOWN!
74

  
75
  - STUD_ST407:
76
      x: UNKNOWN!
77
      y: UNKNOWN!
78

  
79
  - M16:
80
      x: UNKNOWN!
81
      y: UNKNOWN!
82

  
83
  - Zavora-Kaplirova:
84
      x: UNKNOWN!
85
      y: UNKNOWN!
86

  
87
  - Zavora-FDU:
88
      x: UNKNOWN!
89
      y: UNKNOWN!
90

  
91
  - KolaBory-vnejsi:
92
      x: UNKNOWN!
93
      y: UNKNOWN!
94

  
95
  - STUD_VC53:
96
      x: UNKNOWN!
97
      y: UNKNOWN!
98

  
99
  - Menza4-kasa4:
100
      x: UNKNOWN!
101
      y: UNKNOWN!
102

  
103
  - KL-Posilovna:
104
      x: UNKNOWN!
105
      y: UNKNOWN!
106

  
107
  - VC-VYJEZD:
108
      x: UNKNOWN!
109
      y: UNKNOWN!
110

  
111
  - EXT/kola-B:
112
      x: UNKNOWN!
113
      y: UNKNOWN!
114

  
115
  - Menza4-kasa5:
116
      x: UNKNOWN!
117
      y: UNKNOWN!
118

  
119
  - L1L2-vchod:
120
      x: UNKNOWN!
121
      y: UNKNOWN!
122

  
123
  - VC-VJEZD:
124
      x: UNKNOWN!
125
      y: UNKNOWN!
126

  
127
  - A3:
128
      x: UNKNOWN!
129
      y: UNKNOWN!
130

  
131
  - STUD_UB113:
132
      x: UNKNOWN!
133
      y: UNKNOWN!
134

  
135
  - STUD_CHEB:
136
      x: UNKNOWN!
137
      y: UNKNOWN!
138

  
139
  - STUD_PRA1:
140
      x: UNKNOWN!
141
      y: UNKNOWN!
142

  
143
  - L1:
144
      x: UNKNOWN!
145
      y: UNKNOWN!
146

  
147
  - STUD_UB211:
148
      x: UNKNOWN!
149
      y: UNKNOWN!
150

  
151
  - A1:
152
      x: UNKNOWN!
153
      y: UNKNOWN!
154

  
155
  - EXT/kola:
156
      x: UNKNOWN!
157
      y: UNKNOWN!
158

  
159
  - STUD_KL87:
160
      x: UNKNOWN!
161
      y: UNKNOWN!
162

  
163
  - UV1-Bufet:
164
      x: UNKNOWN!
165
      y: UNKNOWN!
166

  
167
  - M14:
168
      x: UNKNOWN!
169
      y: UNKNOWN!
170

  
171
  - Zavora-NTIS-vyjezd:
172
      x: UNKNOWN!
173
      y: UNKNOWN!
174

  
175
  - B3-kolarna:
176
      x: UNKNOWN!
177
      y: UNKNOWN!
178

  
179
  - B3-LEVY:
180
      x: UNKNOWN!
181
      y: UNKNOWN!
182

  
183
  - MenzaKL-vydej:
184
      x: UNKNOWN!
185
      y: UNKNOWN!
186

  
187
  - A2-Hlavni vchod:
188
      x: UNKNOWN!
189
      y: UNKNOWN!
190

  
191
  - Parkoviste-vyjezd:
192
      x: UNKNOWN!
193
      y: UNKNOWN!
194

  
modules/crawler/DatasetConfigs/KOLOBEZKY.yaml
1
# jmeno datasetu, pod kterym bude zobrazen v aplikaci
2
dataset-name: KOLOBEZKY
3
# root slozka, ktera obsahuje odkazy na dataset
4
url: https://openstore.zcu.cz/
5
# volitelny parameter, ktery specifikuje vzor jmena datasetu, ktera se budou stahovat
6
regex: OD_ZCU_KOLOBEZKY_[0-9][0-9]_[0-9][0-9][0-9][0-9]_CSV.zip
7
# volitelny parametr, ktery udava jednou za kolik dní se budou nová data vyhledávat
8
update-period: 24
9
# pozice jednotlivych zarizeni, ktera jsou v datasetu
10
devices:
11
  - stojan-knihovna:
12
      x: 49.725145
13
      y: 13.353319
14

  
15
  - stojan-Machovka:
16
      x: 49.725197
17
      y: 13.368016
18

  
19
  -  stojan-FDU:
20
      x: 49.725360
21
      y: 13.347909
22

  
23
  - stojan-bory:
24
      x: 49.724576
25
      y: 13.350861
26

  
27
  - stojan-zcu:
28
      x: 49.726033
29
      y: 13.365958
30

  
31
  - stojan-borska:
32
      x: 49.734518
33
      y: 13.359475
modules/crawler/DatasetConfigs/WIFI.yaml
1
# jmeno datasetu, pod kterym bude zobrazen v aplikaci
2
dataset-name: WIFI
3
# root slozka, ktera obsahuje odkazy na dataset
4
url: https://openstore.zcu.cz/
5
# volitelny parameter, ktery specifikuje vzor jmena datasetu, ktera se budou stahovat
6
regex: OD_ZCU_WIFI_[0-9][0-9]_[0-9][0-9][0-9][0-9]_CSV.zip
7
# volitelny parametr, ktery udava jednou za kolik dní se budou nová data vyhledávat
8
update-period: 24
9
# pozice jednotlivych zarizeni, ktera jsou v datasetu
10
devices:
11
  - ul-s3-cat2960-sw.zcu.cz:
12
      x: UNKNOWN!
13
      y: UNKNOWN!
14

  
15
  - bk-a1-s6-cat2960-sw.bk.zcu.cz:
16
      x: UNKNOWN!
17
      y: UNKNOWN!
18

  
19
  - lk-l2-s1-sw.lk.zcu.cz:
20
      x: UNKNOWN!
21
      y: UNKNOWN!
22

  
23
  - mk-b36-s3-sw.mk.zcu.cz:
24
      x: UNKNOWN!
25
      y: UNKNOWN!
26

  
27
  - lk-l1-s3-sw.lk.zcu.cz:
28
      x: UNKNOWN!
29
      y: UNKNOWN!
30

  
31
  - bk-a1-s4-sw.bk.zcu.cz:
32
      x: UNKNOWN!
33
      y: UNKNOWN!
34

  
35
  - kl-idf1-cat2960-sw.zcu.cz:
36
      x: UNKNOWN!
37
      y: UNKNOWN!
38

  
39
  - mk-b36-s8-sw.mk.zcu.cz:
40
      x: UNKNOWN!
41
      y: UNKNOWN!
42

  
43
  - mk-k-s3-sw.mk.zcu.cz:
44
      x: UNKNOWN!
45
      y: UNKNOWN!
46

  
47
  - ul-kks-s2-cat2960-sw.zcu.cz:
48
      x: UNKNOWN!
49
      y: UNKNOWN!
50

  
51
  - lk-s2-cat2960-sw.zcu.cz:
52
      x: UNKNOWN!
53
      y: UNKNOWN!
54

  
55
  - kl-s2-idf1-cat2960-sw.zcu.cz:
56
      x: UNKNOWN!
57
      y: UNKNOWN!
58

  
59
  - kl-idf3-cat2960-sw.zcu.cz:
60
      x: UNKNOWN!
61
      y: UNKNOWN!
62

  
63
  - mk-m20-s11-sw.mk.zcu.cz:
64
      x: UNKNOWN!
65
      y: UNKNOWN!
66

  
67
  - ec-s1-cat4506-sw.zcu.cz:
68
      x: UNKNOWN!
69
      y: UNKNOWN!
70

  
71
  - mk-k-cat2960-sw.zcu.cz:
72
      x: UNKNOWN!
73
      y: UNKNOWN!
74

  
75
  - ic-n1-nx5548-sw.zcu.cz:
76
      x: UNKNOWN!
77
      y: UNKNOWN!
78

  
79
  - tf-s3-cat2960-sw.zcu.cz:
80
      x: UNKNOWN!
81
      y: UNKNOWN!
82

  
83
  - um-cat2960-sw.zcu.cz:
84
      x: UNKNOWN!
85
      y: UNKNOWN!
86

  
87
  - ic-ui411-cat2960-sw.zcu.cz:
88
      x: UNKNOWN!
89
      y: UNKNOWN!
90

  
91
  - lk-s1-cat2960-sw.zcu.cz:
92
      x: UNKNOWN!
93
      y: UNKNOWN!
94

  
95
  - ur-cat2960-sw.zcu.cz:
96
      x: UNKNOWN!
97
      y: UNKNOWN!
98

  
99
  - mk-b36-s2-sw.mk.zcu.cz:
100
      x: UNKNOWN!
101
      y: UNKNOWN!
102

  
103
  - el-cat2960-sw.zcu.cz:
104
      x: UNKNOWN!
105
      y: UNKNOWN!
106

  
107
  - ek-s2-cat4506-sw.zcu.cz:
108
      x: UNKNOWN!
109
      y: UNKNOWN!
110

  
111
  - lk-l1-s8-sw.lk.zcu.cz:
112
      x: UNKNOWN!
113
      y: UNKNOWN!
114

  
115
  - ic-n2-nx5548-sw.zcu.cz:
116
      x: UNKNOWN!
117
      y: UNKNOWN!
118

  
119
  - kl-s1-cat4506-sw.zcu.cz:
120
      x: UNKNOWN!
121
      y: UNKNOWN!
122

  
123
  - eh-cat2960-sw.zcu.cz:
124
      x: UNKNOWN!
125
      y: UNKNOWN!
126

  
127
  - mk-m20-s8-sw.mk.zcu.cz:
128
      x: UNKNOWN!
129
      y: UNKNOWN!
130

  
131
  - rj-s1-cat2960-sw.zcu.cz:
132
      x: UNKNOWN!
133
      y: UNKNOWN!
134

  
135
  - lk-l2-s6-sw.lk.zcu.cz:
136
      x: UNKNOWN!
137
      y: UNKNOWN!
138

  
139
  - mk-m16-s3-sw.mk.zcu.cz:
140
      x: UNKNOWN!
141
      y: UNKNOWN!
142

  
143
  - ec-s2-cat4506-sw.zcu.cz:
144
      x: UNKNOWN!
145
      y: UNKNOWN!
146

  
147
  - us-idf5-cat4506-sw.zcu.cz:
148
      x: UNKNOWN!
149
      y: UNKNOWN!
150

  
151
  - lk-l1-cat2960-sw.lk.zcu.cz:
152
      x: UNKNOWN!
153
      y: UNKNOWN!
154

  
155
  - ul-s8-cat3560-sw.zcu.cz:
156
      x: UNKNOWN!
157
      y: UNKNOWN!
158

  
159
  - mk-m16-s4-sw.mk.zcu.cz:
160
      x: UNKNOWN!
161
      y: UNKNOWN!
162

  
163
  - eu-s1-cat2960-sw.zcu.cz:
164
      x: UNKNOWN!
165
      y: UNKNOWN!
166

  
167
  - mk-m20-s3-sw.mk.zcu.cz:
168
      x: UNKNOWN!
169
      y: UNKNOWN!
170

  
171
  - el-z1-cat2960c-sw.zcu.cz:
172
      x: UNKNOWN!
173
      y: UNKNOWN!
174

  
175
  - uv-cat2960-sw.zcu.cz:
176
      x: UNKNOWN!
177
      y: UNKNOWN!
178

  
179
  - ic-s1-cat4507-sw.zcu.cz:
180
      x: UNKNOWN!
181
      y: UNKNOWN!
182

  
183
  - lk-s3-cat2960-sw.zcu.cz:
184
      x: UNKNOWN!
185
      y: UNKNOWN!
186

  
187
  - rj-cat2960-sw.zcu.cz:
188
      x: UNKNOWN!
189
      y: UNKNOWN!
190

  
191
  - so-cat4506-sw.zcu.cz:
192
      x: UNKNOWN!
193
      y: UNKNOWN!
194

  
195
  - un-idf3b-cat4510-sw.zcu.cz:
196
      x: UNKNOWN!
197
      y: UNKNOWN!
198

  
199
  - mk-b36-s4-sw.mk.zcu.cz:
200
      x: UNKNOWN!
201
      y: UNKNOWN!
202

  
203
  - bk-a1-s3-sw.bk.zcu.cz:
204
      x: UNKNOWN!
205
      y: UNKNOWN!
206

  
207
  - jj-s1-cat2960-sw.zcu.cz:
208
      x: UNKNOWN!
209
      y: UNKNOWN!
210

  
211
  - mk-cat4900-sw.zcu.cz:
212
      x: UNKNOWN!
213
      y: UNKNOWN!
214

  
215
  - mk-m14-s2-sw.mk.zcu.cz:
216
      x: UNKNOWN!
217
      y: UNKNOWN!
218

  
219
  - el-z2-cat2960c-sw.zcu.cz:
220
      x: UNKNOWN!
221
      y: UNKNOWN!
222

  
223
  - tc-s4-cat2960-sw.zcu.cz:
224
      x: UNKNOWN!
225
      y: UNKNOWN!
226

  
227
  - uh-cat2960-sw.zcu.cz:
228
      x: UNKNOWN!
229
      y: UNKNOWN!
230

  
231
  - uk-rek-cat2960-sw.zcu.cz:
232
      x: UNKNOWN!
233
      y: UNKNOWN!
234

  
235
  - bk-a2-s1-sw.bk.zcu.cz:
236
      x: UNKNOWN!
237
      y: UNKNOWN!
238

  
239
  - vc-s2-cat2960-sw.zcu.cz:
240
      x: UNKNOWN!
241
      y: UNKNOWN!
242

  
243
  - mk-m14-s4-sw.mk.zcu.cz:
244
      x: UNKNOWN!
245
      y: UNKNOWN!
246

  
247
  - uk-s1-cat2960-sw.zcu.cz:
248
      x: UNKNOWN!
249
      y: UNKNOWN!
250

  
251
  - tc-s2-cat2960-sw.zcu.cz:
252
      x: UNKNOWN!
253
      y: UNKNOWN!
254

  
255
  - so-s1-cat2960-sw.zcu.cz:
256
      x: UNKNOWN!
257
      y: UNKNOWN!
258

  
259
  - lk-l2-s3-sw.lk.zcu.cz:
260
      x: UNKNOWN!
261
      y: UNKNOWN!
262

  
263
  - ic-2504-wlc:
264
      x: UNKNOWN!
265
      y: UNKNOWN!
266

  
267
  - lk-l1-s5-sw.lk.zcu.cz:
268
      x: UNKNOWN!
269
      y: UNKNOWN!
270

  
271
  - bk-a2-s2-sw.bk.zcu.cz:
272
      x: UNKNOWN!
273
      y: UNKNOWN!
274

  
275
  - tf-cat4500-sw.zcu.cz:
276
      x: UNKNOWN!
277
      y: UNKNOWN!
278

  
279
  - mk-m20-s1-sw.mk.zcu.cz:
280
      x: UNKNOWN!
281
      y: UNKNOWN!
282

  
283
  - bk-a3-s6-cat2960-sw.bk.zcu.cz:
284
      x: UNKNOWN!
285
      y: UNKNOWN!
286

  
287
  - uk-kks-cat2960-sw.zcu.cz:
288
      x: UNKNOWN!
289
      y: UNKNOWN!
290

  
291
  - ul-rti-cat2960-sw.zcu.cz:
292
      x: UNKNOWN!
293
      y: UNKNOWN!
294

  
295
  - mk-m20-s12-sw.mk.zcu.cz:
296
      x: UNKNOWN!
297
      y: UNKNOWN!
298

  
299
  - th-s1-cat3750-sw.zcu.cz:
300
      x: UNKNOWN!
301
      y: UNKNOWN!
302

  
303
  - bk-a3-cat2960-sw.bk.zcu.cz:
304
      x: UNKNOWN!
305
      y: UNKNOWN!
306

  
307
  - ul-s-cat6506-sw.zcu.cz:
308
      x: UNKNOWN!
309
      y: UNKNOWN!
310

  
311
  - un-idf4a-cat4510-sw.zcu.cz:
312
      x: UNKNOWN!
313
      y: UNKNOWN!
314

  
315
  - mk-b36-cat3560-sw.zcu.cz:
316
      x: UNKNOWN!
317
      y: UNKNOWN!
318

  
319
  - st-cat2960-sw.zcu.cz:
320
      x: UNKNOWN!
321
      y: UNKNOWN!
322

  
323
  - ic-4404-wlc:
324
      x: UNKNOWN!
325
      y: UNKNOWN!
326

  
327
  - mk-m20-s10-sw.mk.zcu.cz:
328
      x: UNKNOWN!
329
      y: UNKNOWN!
330

  
331
  - tg-s2-cat3560-sw.zcu.cz:
332
      x: UNKNOWN!
333
      y: UNKNOWN!
334

  
335
  - tc-s1-cat2960-sw.zcu.cz:
336
      x: UNKNOWN!
337
      y: UNKNOWN!
338

  
339
  - mk-k-s4-sw.mk.zcu.cz:
340
      x: UNKNOWN!
341
      y: UNKNOWN!
342

  
343
  - nc-s1-cat3560-sw.zcu.cz:
344
      x: UNKNOWN!
345
      y: UNKNOWN!
346

  
347
  - tf-s1-cat2960-sw.zcu.cz:
348
      x: UNKNOWN!
349
      y: UNKNOWN!
350

  
351
  - vc-s2-idf-cat2960-sw.zcu.cz:
352
      x: UNKNOWN!
353
      y: UNKNOWN!
354

  
355
  - vc-s1-idf-cat2960-sw.zcu.cz:
356
      x: UNKNOWN!
357
      y: UNKNOWN!
358

  
359
  - kl-s2-cat2960-sw.zcu.cz:
360
      x: UNKNOWN!
361
      y: UNKNOWN!
362

  
363
  - ko-s1-cat2960-sw.zcu.cz:
364
      x: UNKNOWN!
365
      y: UNKNOWN!
366

  
367
  - ul-kks-s1-cat2960-sw.zcu.cz:
368
      x: UNKNOWN!
369
      y: UNKNOWN!
370

  
371
  - sp-cat4506-sw.zcu.cz:
372
      x: UNKNOWN!
373
      y: UNKNOWN!
374

  
375
  - bk-a3-s4-sw.bk.zcu.cz:
376
      x: UNKNOWN!
377
      y: UNKNOWN!
378

  
379
  - mk-m16-s5-sw.mk.zcu.cz:
380
      x: UNKNOWN!
381
      y: UNKNOWN!
382

  
383
  - th-cat2960-sw.zcu.cz:
384
      x: UNKNOWN!
385
      y: UNKNOWN!
386

  
387
  - uu-kks-cat2960-sw.zcu.cz:
388
      x: UNKNOWN!
389
      y: UNKNOWN!
390

  
391
  - mk-m20-s5-sw.mk.zcu.cz:
392
      x: UNKNOWN!
393
      y: UNKNOWN!
394

  
395
  - mk-m20-s9-sw.mk.zcu.cz:
396
      x: UNKNOWN!
397
      y: UNKNOWN!
398

  
399
  - ul-kks-s3-cat2960-sw.zcu.cz:
400
      x: UNKNOWN!
401
      y: UNKNOWN!
402

  
403
  - tg-cat2960-sw.zcu.cz:
404
      x: UNKNOWN!
405
      y: UNKNOWN!
406

  
407
  - ul-tel2-cat2960-sw.zcu.cz:
408
      x: UNKNOWN!
409
      y: UNKNOWN!
410

  
411
  - ul-tel-cat2960-sw.zcu.cz:
412
      x: UNKNOWN!
413
      y: UNKNOWN!
414

  
415
  - ic-u5-cat2960-sw.zcu.cz:
416
      x: UNKNOWN!
417
      y: UNKNOWN!
418

  
419
  - lk-l2-s2-sw.lk.zcu.cz:
420
      x: UNKNOWN!
421
      y: UNKNOWN!
422

  
423
  - tg-s1-cat3560-sw.zcu.cz:
424
      x: UNKNOWN!
425
      y: UNKNOWN!
426

  
427
  - mk-m20-s7-sw.mk.zcu.cz:
428
      x: UNKNOWN!
429
      y: UNKNOWN!
430

  
431
  - kl-idf2-cat2960-sw.zcu.cz:
432
      x: UNKNOWN!
433
      y: UNKNOWN!
434

  
435
  - uf-cat2960-sw.zcu.cz:
436
      x: UNKNOWN!
437
      y: UNKNOWN!
438

  
439
  - ek-s3-cat2960-sw.zcu.cz:
440
      x: UNKNOWN!
441
      y: UNKNOWN!
442

  
443
  - mk-b36-s1-sw.mk.zcu.cz:
444
      x: UNKNOWN!
445
      y: UNKNOWN!
446

  
447
  - mk-m14-s1-sw.mk.zcu.cz:
448
      x: UNKNOWN!
449
      y: UNKNOWN!
450

  
451
  - un-idf3a-cat4510-sw.zcu.cz:
452
      x: UNKNOWN!
453
      y: UNKNOWN!
454

  
455
  - bk-a1-s5-sw.bk.zcu.cz:
456
      x: UNKNOWN!
457
      y: UNKNOWN!
458

  
459
  - vc-s3-cat2960-sw.zcu.cz:
460
      x: UNKNOWN!
461
      y: UNKNOWN!
462

  
463
  - uk-kpv-cat2960-sw.zcu.cz:
464
      x: UNKNOWN!
465
      y: UNKNOWN!
466

  
467
  - ut-cat2960-sw.zcu.cz:
468
      x: UNKNOWN!
469
      y: UNKNOWN!
470

  
471
  - uc-idf2a-cat4510-sw.zcu.cz:
472
      x: UNKNOWN!
473
      y: UNKNOWN!
474

  
475
  - ps-cat2960-sw.zcu.cz:
476
      x: UNKNOWN!
477
      y: UNKNOWN!
478

  
479
  - mk-m16-cat3560-sw.mk.zcu.cz:
480
      x: UNKNOWN!
481
      y: UNKNOWN!
482

  
483
  - mk-b36-s7-sw.mk.zcu.cz:
484
      x: UNKNOWN!
485
      y: UNKNOWN!
486

  
487
  - ek-s1-cat4506-sw.zcu.cz:
488
      x: UNKNOWN!
489
      y: UNKNOWN!
490

  
491
  - bk-a3-s3-sw.bk.zcu.cz:
492
      x: UNKNOWN!
493
      y: UNKNOWN!
494

  
495
  - uk-kto-cat2960-sw.zcu.cz:
496
      x: UNKNOWN!
497
      y: UNKNOWN!
498

  
499
  - tf-s2-cat3560-sw.zcu.cz:
500
      x: UNKNOWN!
501
      y: UNKNOWN!
502

  
503
  - ts-cat4900-sw.zcu.cz:
504
      x: UNKNOWN!
505
      y: UNKNOWN!
506

  
507
  - cd-s8-cat2960-sw.zcu.cz:
508
      x: UNKNOWN!
509
      y: UNKNOWN!
510

  
511
  - mk-m14-s3-sw.mk.zcu.cz:
512
      x: UNKNOWN!
513
      y: UNKNOWN!
514

  
515
  - uk-ops-cat2960-sw.zcu.cz:
516
      x: UNKNOWN!
517
      y: UNKNOWN!
518

  
519
  - cd-s2-cat2960-sw.zcu.cz:
520
      x: UNKNOWN!
521
      y: UNKNOWN!
522

  
523
  - bk-a3-s1-sw.bk.zcu.cz:
524
      x: UNKNOWN!
525
      y: UNKNOWN!
526

  
527
  - ic-ui411-s2-cat3560-sw.zcu.cz:
528
      x: UNKNOWN!
529
      y: UNKNOWN!
530

  
531
  - cd-s7-cat2960-sw.zcu.cz:
532
      x: UNKNOWN!
533
      y: UNKNOWN!
534

  
535
  - bk-a3-s2-sw.bk.zcu.cz:
536
      x: UNKNOWN!
537
      y: UNKNOWN!
538

  
539
  - un-idf4b-cat4510-sw.zcu.cz:
540
      x: UNKNOWN!
541
      y: UNKNOWN!
542

  
543
  - ul-s4-cat2960-sw.zcu.cz:
544
      x: UNKNOWN!
545
      y: UNKNOWN!
546

  
547
  - mk-b36-s6-sw.mk.zcu.cz:
548
      x: UNKNOWN!
549
      y: UNKNOWN!
550

  
551
  - us-z-cat2960c-sw.zcu.cz:
552
      x: UNKNOWN!
553
      y: UNKNOWN!
554

  
555
  - ls-s1-cat4510-sw.zcu.cz:
556
      x: UNKNOWN!
557
      y: UNKNOWN!
558

  
559
  - bk-a2-cat2960-sw.bk.zcu.cz:
560
      x: UNKNOWN!
561
      y: UNKNOWN!
562

  
563
  - mk-b36-s11-sw.mk.zcu.cz:
564
      x: UNKNOWN!
565
      y: UNKNOWN!
566

  
567
  - mk-k-s1-sw.mk.zcu.cz:
568
      x: UNKNOWN!
569
      y: UNKNOWN!
570

  
571
  - tc-s6-cat2960-sw.zcu.cz:
572
      x: UNKNOWN!
573
      y: UNKNOWN!
574

  
575
  - ic-5520-wlc:
576
      x: UNKNOWN!
577
      y: UNKNOWN!
578

  
579
  - eu-cat2960-sw.zcu.cz:
580
      x: UNKNOWN!
581
      y: UNKNOWN!
582

  
583
  - rs-cat2960-sw.zcu.cz:
584
      x: UNKNOWN!
585
      y: UNKNOWN!
586

  
587
  - lk-l2-s5-sw.lk.zcu.cz:
588
      x: UNKNOWN!
589
      y: UNKNOWN!
590

  
591
  - ic-ui402-cat2960-sw.zcu.cz:
592
      x: UNKNOWN!
593
      y: UNKNOWN!
594

  
595
  - tc-s3-cat2960-sw.zcu.cz:
596
      x: UNKNOWN!
597
      y: UNKNOWN!
598

  
599
  - cd-s6-cat2960-sw.zcu.cz:
600
      x: UNKNOWN!
601
      y: UNKNOWN!
602

  
603
  - uu-cat4506-sw.zcu.cz:
604
      x: UNKNOWN!
605
      y: UNKNOWN!
606

  
607
  - mk-m20-s6-sw.mk.zcu.cz:
608
      x: UNKNOWN!
609
      y: UNKNOWN!
610

  
611
  - cd-cat4500-sw.zcu.cz:
612
      x: UNKNOWN!
613
      y: UNKNOWN!
614

  
615
  - uc-idf1b-cat4510-sw.zcu.cz:
616
      x: UNKNOWN!
617
      y: UNKNOWN!
618

  
619
  - cd-s1-cat2960-sw.zcu.cz:
620
      x: UNKNOWN!
621
      y: UNKNOWN!
622

  
623
  - lk-l1-s7-sw.lk.zcu.cz:
624
      x: UNKNOWN!
625
      y: UNKNOWN!
626

  
627
  - ic-5508-wlc:
628
      x: UNKNOWN!
629
      y: UNKNOWN!
630

  
631
  - mk-m20-s2-sw.mk.zcu.cz:
632
      x: UNKNOWN!
633
      y: UNKNOWN!
634

  
635
  - ic-ui403-cat2960-sw.zcu.cz:
636
      x: UNKNOWN!
637
      y: UNKNOWN!
638

  
639
  - ep-cat2960-sw.zcu.cz:
640
      x: UNKNOWN!
641
      y: UNKNOWN!
642

  
643
  - pc-s1-cat4506-sw.zcu.cz:
644
      x: UNKNOWN!
645
      y: UNKNOWN!
646

  
647
  - mk-k-s5-cat2960-sw.mk.zcu.cz:
648
      x: UNKNOWN!
649
      y: UNKNOWN!
650

  
651
  - uc-kma-cat2960-sw.zcu.cz:
652
      x: UNKNOWN!
653
      y: UNKNOWN!
654

  
655
  - bk-a3-s5-sw.bk.zcu.cz:
656
      x: UNKNOWN!
657
      y: UNKNOWN!
658

  
659
  - cd-s4-cat2960-sw.zcu.cz:
660
      x: UNKNOWN!
661
      y: UNKNOWN!
662

  
663
  - bk-a2-s3-sw.bk.zcu.cz:
664
      x: UNKNOWN!
665
      y: UNKNOWN!
666

  
667
  - cd-s9-cat3750-sw.zcu.cz:
668
      x: UNKNOWN!
669
      y: UNKNOWN!
670

  
671
  - ul-s2-cat4506-sw.zcu.cz:
672
      x: UNKNOWN!
673
      y: UNKNOWN!
674

  
675
  - mk-b36-s5-sw.mk.zcu.cz:
676
      x: UNKNOWN!
677
      y: UNKNOWN!
678

  
679
  - uc-idf2b-cat4510-sw.zcu.cz:
680
      x: UNKNOWN!
681
      y: UNKNOWN!
682

  
683
  - ls-z-cat2960c-sw.zcu.cz:
684
      x: UNKNOWN!
685
      y: UNKNOWN!
686

  
687
  - lk-l1-s4-sw.lk.zcu.cz:
688
      x: UNKNOWN!
689
      y: UNKNOWN!
690

  
691
  - lk-l2-s4-sw.lk.zcu.cz:
692
      x: UNKNOWN!
693
      y: UNKNOWN!
694

  
695
  - ub-cat4506-sw.zcu.cz:
696
      x: UNKNOWN!
697
      y: UNKNOWN!
698

  
699
  - uc-idf1a-cat4510-sw.zcu.cz:
700
      x: UNKNOWN!
701
      y: UNKNOWN!
702

  
703
  - ul-s1-cat4506-sw.zcu.cz:
704
      x: UNKNOWN!
705
      y: UNKNOWN!
706

  
707
  - ps-s2-cat2960-sw.zcu.cz:
708
      x: UNKNOWN!
709
      y: UNKNOWN!
710

  
711
  - mk-m20-s4-sw.mk.zcu.cz:
712
      x: UNKNOWN!
713
      y: UNKNOWN!
714

  
715
  - so-s2-cat2960-sw.zcu.cz:
716
      x: UNKNOWN!
717
      y: UNKNOWN!
718

  
719
  - mk-b36-s9-sw.mk.zcu.cz:
720
      x: UNKNOWN!
721
      y: UNKNOWN!
722

  
723
  - bk-a2-s5-cat2960-sw.bk.zcu.cz:
724
      x: UNKNOWN!
725
      y: UNKNOWN!
726

  
727
  - lk-l1-s1-sw.lk.zcu.cz:
728
      x: UNKNOWN!
729
      y: UNKNOWN!
730

  
731
  - mk-m16-s1-sw.mk.zcu.cz:
732
      x: UNKNOWN!
733
      y: UNKNOWN!
734

  
735
  - mk-m16-s2-sw.mk.zcu.cz:
736
      x: UNKNOWN!
737
      y: UNKNOWN!
738

  
739
  - lk-l1-s2-sw.lk.zcu.cz:
740
      x: UNKNOWN!
741
      y: UNKNOWN!
742

  
743
  - lk-l2-s7-sw.lk.zcu.cz:
744
      x: UNKNOWN!
745
      y: UNKNOWN!
746

  
747
  - cd-s3-cat2960-sw.zcu.cz:
748
      x: UNKNOWN!
749
      y: UNKNOWN!
750

  
751
  - mk-k-s2-sw.mk.zcu.cz:
752
      x: UNKNOWN!
753
      y: UNKNOWN!
754

  
755
  - ls-s2-cat4506-sw.zcu.cz:
756
      x: UNKNOWN!
757
      y: UNKNOWN!
758

  
759
  - bk-cat3750-sw.zcu.cz:
760
      x: UNKNOWN!
761
      y: UNKNOWN!
762

  
763
  - lk-l1-s6-sw.lk.zcu.cz:
764
      x: UNKNOWN!
765
      y: UNKNOWN!
766

  
767
  - ic-u1-cat2960-sw.zcu.cz:
768
      x: UNKNOWN!
769
      y: UNKNOWN!
770

  
771
  - hj-s1-cat4506-sw.zcu.cz:
772
      x: UNKNOWN!
773
      y: UNKNOWN!
774

  
775
  - bk-a4-cat2960-sw.bk.zcu.cz:
776
      x: UNKNOWN!
777
      y: UNKNOWN!
778

  
779
  - uk-cat2960-sw.zcu.cz:
780
      x: UNKNOWN!
781
      y: UNKNOWN!
782

  
783
  - rti-cat4506-sw.zcu.cz:
784
      x: UNKNOWN!
785
      y: UNKNOWN!
786

  
787
  - tc-s5-cat2960-sw.zcu.cz:
788
      x: UNKNOWN!
789
      y: UNKNOWN!
790

  
791
  - mk-m14-cat2960-sw.mk.zcu.cz:
792
      x: UNKNOWN!
793
      y: UNKNOWN!
794

  
795
  - mk-b36-s10-sw.mk.zcu.cz:
796
      x: UNKNOWN!
797
      y: UNKNOWN!
798

  
799
  - ul-s6-cat2960-sw.zcu.cz:
800
      x: UNKNOWN!
801
      y: UNKNOWN!
802

  
803
  - ic-ui121-cat3560-sw.zcu.cz:
804
      x: UNKNOWN!
805
      y: UNKNOWN!
806

  
807
  - bk-a1-s2-sw.bk.zcu.cz:
808
      x: UNKNOWN!
809
      y: UNKNOWN!
810

  
811
  - ls-s3-cat2960-sw.zcu.cz:
812
      x: UNKNOWN!
813
      y: UNKNOWN!
814

  
815
  - ic-u3-cat2960-sw.zcu.cz:
816
      x: UNKNOWN!
817
      y: UNKNOWN!
818

  
819
  - bk-a1-s1-sw.bk.zcu.cz:
820
      x: UNKNOWN!
821
      y: UNKNOWN!
822

  
823
  - ic-ui411-s1-cat3560-sw.zcu.cz:
824
      x: UNKNOWN!
825
      y: UNKNOWN!
826

  
827
  - rj-s2-cat2960-sw.zcu.cz:
828
      x: UNKNOWN!
829
      y: UNKNOWN!
830

  
831
  - ko-cat3560-sw.zcu.cz:
832
      x: UNKNOWN!
833
      y: UNKNOWN!
834

  
835
  - jj-s3-cat2960-sw.zcu.cz:
836
      x: UNKNOWN!
837
      y: UNKNOWN!
838

  
839
  - cd-s5-cat2960-sw.zcu.cz:
840
      x: UNKNOWN!
841
      y: UNKNOWN!
842

  
843
  - vc-s3-idf-cat2960-sw.zcu.cz:
844
      x: UNKNOWN!
845
      y: UNKNOWN!
846

  
modules/crawler/DatasetCrawler/JISCrawler.py
1
from Utilities import FolderProcessor
2
from Utilities.Crawler import BasicCrawlerFunctions
3

  
4
# Path to crawled data
5
CRAWLED_DATA_PATH = "CrawledData/"
6

  
7

  
8
def crawl(config):
9
    """
10
    Implement crawl method that downloads new data to path_for_files
11
    For keeping the project structure
12
    url , regex, and dataset_name from config
13
    You can use already implemented functions from Utilities/Crawler/BasicCrawlerFunctions.py
14

  
15
    Args:
16
        config: loaded configuration file of dataset
17
    """
18
    dataset_name = config["dataset-name"]
19
    url = config['url']
20
    regex = config['regex']
21
    path_for_files = CRAWLED_DATA_PATH + dataset_name + '/'
22

  
23
    first_level_links = BasicCrawlerFunctions.get_all_links(url)
24
    filtered_first_level_links = BasicCrawlerFunctions.filter_links(first_level_links, "^OD_ZCU")
25
    absolute_first_level_links = BasicCrawlerFunctions.create_absolute_links(filtered_first_level_links, url)
26

  
27
    files = []
28

  
29
    for link in absolute_first_level_links:
30
        second_level_links = BasicCrawlerFunctions.get_all_links(link)
31
        filtered_second_level_links = BasicCrawlerFunctions.filter_links(second_level_links, regex)
32
        absolute_second_level_links = BasicCrawlerFunctions.create_absolute_links(filtered_second_level_links, link)
33
        final_links = BasicCrawlerFunctions.remove_downloaded_links(absolute_second_level_links, dataset_name)
34

  
35
        for file_link in final_links:
36
            files.append(file_link)
37

  
38
    for file in files:
39
        BasicCrawlerFunctions.download_file_from_url(file, dataset_name)
40

  
41
    FolderProcessor.unzip_all_csv_zip_files_in_folder(path_for_files)
modules/crawler/DatasetCrawler/KOLOBEZKYCrawler.py
1
from Utilities import FolderProcessor
2
from Utilities.Crawler import BasicCrawlerFunctions
3

  
4
# Path to crawled data
5
CRAWLED_DATA_PATH = "CrawledData/"
6

  
7

  
8
def crawl(config):
9
    """
10
    Implement crawl method that downloads new data to path_for_files
11
    For keeping the project structure
12
    url , regex, and dataset_name from config
13
    You can use already implemented functions from Utilities/Crawler/BasicCrawlerFunctions.py
14

  
15
    Args:
16
        config: loaded configuration file of dataset
17
    """
18
    dataset_name = config["dataset-name"]
19
    url = config['url']
20
    regex = config['regex']
21
    path_for_files = CRAWLED_DATA_PATH + dataset_name + '/'
22

  
23
    first_level_links = BasicCrawlerFunctions.get_all_links(url)
24
    filtered_first_level_links = BasicCrawlerFunctions.filter_links(first_level_links, "^OD_ZCU")
25
    absolute_first_level_links = BasicCrawlerFunctions.create_absolute_links(filtered_first_level_links, url)
26

  
27
    files = []
28

  
29
    for link in absolute_first_level_links:
30
        second_level_links = BasicCrawlerFunctions.get_all_links(link)
31
        filtered_second_level_links = BasicCrawlerFunctions.filter_links(second_level_links, regex)
32
        absolute_second_level_links = BasicCrawlerFunctions.create_absolute_links(filtered_second_level_links, link)
33
        final_links = BasicCrawlerFunctions.remove_downloaded_links(absolute_second_level_links, dataset_name)
34

  
35
        for file_link in final_links:
36
            files.append(file_link)
37

  
38
    for file in files:
39
        BasicCrawlerFunctions.download_file_from_url(file, dataset_name)
40

  
41
    FolderProcessor.unzip_all_csv_zip_files_in_folder(path_for_files)
modules/crawler/DatasetCrawler/WIFICrawler.py
1
from Utilities import FolderProcessor
2
from Utilities.Crawler import BasicCrawlerFunctions
3

  
4
# Path to crawled data
5
CRAWLED_DATA_PATH = "CrawledData/"
6

  
7

  
8
def crawl(config):
9
    """
10
    Implement crawl method that downloads new data to path_for_files
11
    For keeping the project structure
12
    url , regex, and dataset_name from config
13
    You can use already implemented functions from Utilities/Crawler/BasicCrawlerFunctions.py
14

  
15
    Args:
16
        config: loaded configuration file of dataset
17
    """
18
    dataset_name = config["dataset-name"]
19
    url = config['url']
20
    regex = config['regex']
21
    path_for_files = CRAWLED_DATA_PATH + dataset_name + '/'
22

  
23
    first_level_links = BasicCrawlerFunctions.get_all_links(url)
24
    filtered_first_level_links = BasicCrawlerFunctions.filter_links(first_level_links, "^OD_ZCU")
25
    absolute_first_level_links = BasicCrawlerFunctions.create_absolute_links(filtered_first_level_links, url)
26

  
27
    files = []
28

  
29
    for link in absolute_first_level_links:
30
        second_level_links = BasicCrawlerFunctions.get_all_links(link)
31
        filtered_second_level_links = BasicCrawlerFunctions.filter_links(second_level_links, regex)
32
        absolute_second_level_links = BasicCrawlerFunctions.create_absolute_links(filtered_second_level_links, link)
33
        final_links = BasicCrawlerFunctions.remove_downloaded_links(absolute_second_level_links, dataset_name)
34

  
35
        for file_link in final_links:
36
            files.append(file_link)
37

  
38
    for file in files:
39
        BasicCrawlerFunctions.download_file_from_url(file, dataset_name)
40

  
41
    FolderProcessor.unzip_all_csv_zip_files_in_folder(path_for_files)
modules/crawler/DatasetProcessing/JISProcessor.py
1
from Utilities.CSV import CSVDataLine, CSVutils
2
from Utilities import DateFormating
3

  
4

  
5
def process_file(filename):
6
    """
7
    Method that take path to crawled file and outputs date dictionary using method:
8
    CSVutils.export_data_to_csv(filename, date_dict)
9
    Date dictionary is a dictionary where keys are dates in format ddmmYYYYhh (0804201815)
10
    and value is dictionary where keys devices (specified in configuration file)
11
    and value is CSVDataLine.CSVDataLine with device,date and occurrence
12

  
13
    Args:
14
        filename: name of processed file
15

  
16
    Returns:
17
        False if not implemented
18
        True when implemented
19
    """
20
    with open(filename, "r", encoding="utf-8") as file:
21

  
22
        date_dict = dict()
23

  
24
        for line in file:
25

  
26
            array = line.split(";")
27

  
28
            date = DateFormating.date_time_formatter(array[1][1:-1])
29
            name = array[0][1:-1]
30
            occurrence = array[2][:-1]
31

  
32
            if date not in date_dict:
33
                date_dict[date] = dict()
34

  
35
            if name in date_dict[date]:
36
                date_dict[date][name].occurrence += int(occurrence)
37
            else:
38
                date_dict[date][name] = CSVDataLine.CSVDataLine(name, date, int(occurrence))
39

  
40
    CSVutils.export_data_to_csv(filename, date_dict)
41
    return True
42

  
modules/crawler/DatasetProcessing/KOLOBEZKYProcessor.py
1
from Utilities.CSV import CSVDataLine, CSVutils
2
from Utilities import DateFormating
3

  
4

  
5
def process_file(filename):
6
    """
7
    Method that take path to crawled file and outputs date dictionary using method:
8
    CSVutils.export_data_to_csv(filename, date_dict)
9
    Date dictionary is a dictionary where keys are dates in format ddmmYYYYhh (0804201815)
10
    and value is dictionary where keys devices (specified in configuration file)
11
    and value is CSVDataLine.CSVDataLine with device,date and occurrence
12

  
13
    Args:
14
        filename: name of processed file
15

  
16
    Returns:
17
        False if not implemented
18
        True when implemented
19
    """
20
    with open(filename, "r") as file:
21

  
22
        date_dict = dict()
23

  
24
        for line in file:
25

  
26
            array = line.split(";")
27

  
28
            date = DateFormating.date_time_formatter(array[0][1:-1])
29
            name = array[1][1:-1]
30

  
31
            if date not in date_dict:
32
                date_dict[date] = dict()
33

  
34
            if name in date_dict[date]:
35
                date_dict[date][name].occurrence += 1
36
            else:
37
                date_dict[date][name] = CSVDataLine.CSVDataLine(name, date, 1)
38

  
39
    CSVutils.export_data_to_csv(filename, date_dict)
40
    return True
41

  
modules/crawler/DatasetProcessing/WIFIProcessor.py
1
from Utilities.CSV import CSVDataLine, CSVutils
2
from Utilities import DateFormating
3

  
4

  
5
def process_file(filename):
6
    """
7
    Method that take path to crawled file and outputs date dictionary using method:
8
    CSVutils.export_data_to_csv(filename, date_dict)
9
    Date dictionary is a dictionary where keys are dates in format ddmmYYYYhh (0804201815)
10
    and value is dictionary where keys devices (specified in configuration file)
11
    and value is CSVDataLine.CSVDataLine with device,date and occurrence
12

  
13
    Args:
14
        filename: name of processed file
15

  
16
    Returns:
17
        False if not implemented
18
        True when implemented
19
    """
20
    with open(filename, "r", encoding="utf-8") as file:
21

  
22
        date_dict = dict()
23

  
24
        for line in file:
25

  
26
            array = line.split(";")
27

  
28
            date = DateFormating.date_time_formatter(array[4][1:-2])
29
            name = array[1][1:-1]
30
            occurrence = array[0]
31

  
32
            if date not in date_dict:
33
                date_dict[date] = dict()
34

  
35
            if name in date_dict[date]:
36
                date_dict[date][name].occurrence += int(occurrence)
37
            else:
38
                date_dict[date][name] = CSVDataLine.CSVDataLine(name, date, int(occurrence))
39

  
40
    CSVutils.export_data_to_csv(filename, date_dict)
41
    return True
42

  
modules/crawler/Pipeline.py
1
from Utilities import FolderProcessor, ConfigureFunctions
2
from Utilities.Database import DatabaseLoader
3

  
4
# Path to crawled data
5
CRAWLED_DATA_PATH = "CrawledData/"
6
# Path to processed data
7
PROCESSED_DATA_PATH = "ProcessedData/"
8
# Path to dataset crawler implementations
9
CRAWLER_LIB_PATH = "DatasetCrawler."
10
# Path to dataset processor implementations
11
PROCESSOR_LIB_PATH = "DatasetProcessing."
12

  
13

  
14
def crawl_data(config):
15
    """
16
      Imports dataset crawler in DatasetCrawler/"dataset_name"Crawler.py
17
      runs crawler.
18

  
19
    Args:
20
        config: loaded configuration file of dataset
21
    """
22
    dataset_name = config["dataset-name"]
23

  
24
    my_function = __import__(CRAWLER_LIB_PATH + dataset_name + "Crawler", globals(), locals(), ['crawl']).crawl
25
    my_function(config)
26

  
27
    dataset_name += '/'
28

  
29

  
30
def process_data(dataset_name):
31
    """
32
    Goes trough every not processed file(not contained in CrawledData/"dataset_name"/ignore.txt)
33
    Imports dataset processor in DatasetProcessing/"dataset_name"Processor.py
34
    Runs processor on every file
35
    After successful processing updates ignore.txt
36

  
37
    Args:
38
        dataset_name: name of dataset that has existing configuration file
39
    """
40
    dataset_path = dataset_name + '/'
41

  
42
    process_file_func = __import__(PROCESSOR_LIB_PATH + dataset_name + "Processor", globals(), locals(),
43
                                   ['process_file']).process_file
44

  
45
    not_processed_files = FolderProcessor.list_of_all_files(CRAWLED_DATA_PATH + dataset_path)
46

  
47
    for not_processed_file in not_processed_files:
48
        process_file_func(CRAWLED_DATA_PATH + dataset_path + not_processed_file)
49
        FolderProcessor.update_ignore_set(CRAWLED_DATA_PATH + dataset_path, not_processed_file)
50

  
51

  
52
def validate_process_data(config):
53
    """
54
    Function goes through newly processed data and checks theirs status
55

  
56
    Args:
57
        config: loaded configuration file of dataset
58

  
59
    Returns:
60
        boolean variable TRUE/FALSE.
61
        Data processed correctly - TRUE
... Rozdílový soubor je zkrácen, protože jeho délka přesahuje max. limit.

Také k dispozici: Unified diff