|
1
|
# Heatmap.ZČU
|
|
2
|
App was created within the school project by ZČU students. The main goal of this project is to create a simple and generic tool for vizualization of heatmap like datasets. The first implementation can be seen at https://heatmap.zcu.cz.
|
|
3
|
## Intro webpage
|
|
4
|
<img src="./doc/images/website-intro.png" style="width: 600px; height: auto;">
|
|
5
|
|
|
6
|
## Heatmap webpage
|
|
7
|
<img src="./doc/images/website-heatmap.png" style="width: 600px; height: auto;">
|
|
8
|
|
|
9
|
## Responsive
|
|
10
|
<img src="./doc/images/website-responsive.png" style="width: 200px; height: auto;">
|
|
11
|
|
|
12
|
<br><hr><br>
|
|
13
|
|
|
14
|
## Technologies
|
|
15
|
|
|
16
|
- [**Nginx**](https://www.nginx.com/) – Webserver
|
|
17
|
- [**Symfony**](https://symfony.com/) – PHP framework
|
|
18
|
- [**MongoDB**](https://www.mongodb.com/) – Database
|
|
19
|
|
|
20
|
<br><hr><br>
|
|
21
|
|
|
22
|
## 🔩 Development env
|
|
23
|
### SW needed for development
|
|
24
|
* **[🐳 docker](https://docs.docker.com/engine/install)**
|
|
25
|
* **[🐳 docker-compose](https://docs.docker.com/compose/install/)**
|
|
26
|
* **[📝 Visual Studio Code](https://code.visualstudio.com/Download)**
|
|
27
|
<br><br>
|
|
28
|
|
|
29
|
### Development setup
|
|
30
|
* First you need to **build** and **run** all needed containers
|
|
31
|
* run either of these scripts inside the **root of the project**:
|
|
32
|
* Linux: `./scripts/dev/build.sh`
|
|
33
|
* Windows: `scripts/dev/build.bat`
|
|
34
|
* check if all containers are running by `docker ps -a`
|
|
35
|
* now if you open **[localhost](https://localhost:80)** you should see working website
|
|
36
|
* Install needed extensions inside vscode
|
|
37
|
* open the project folder with vscode
|
|
38
|
* if you are prompt to install recommended extensions in the right bottom corner click on it (wait for installation to complete)
|
|
39
|
* if **there is no popup for installation** do this: click <img src="./doc/images/extension.png" height="40px" width="40px"/>, <img src="./doc/images/more.png" height="40px" width="40px"/> and choose `Show Recommended extensions` and install extension which is in section **WORKSPACE RECOMMENDATIONS** (`ms-vscode-remote.remote-containers`)
|
|
40
|
* You may be asked to **reload vscode** (blue label inside extension with text)
|
|
41
|
### Symfony (`./website`)
|
|
42
|
* To make changes inside symfony module click on file `symfony.code-workspace` which is in `website` folder
|
|
43
|
* Click on <kbd>Open Workspace</kbd> at the bottom of the screen, wait
|
|
44
|
* Click on <kbd>Reopen in Container</kbd>
|
|
45
|
* Now you are inside the **docker container**. You should see something like this in the bottom left corner
|
|
46
|
|
|
47
|
</br>
|
|
48
|
</br>
|
|
49
|
<img src="./doc/images/dev-container.png">
|
|
50
|
|
|
51
|
* To **close attach or reopen project in container** use these arrows
|
|
52
|
* Now **install recommended extension.**
|
|
53
|
* If you need to run `compose` commands simply open vscode terminal and run your commands there.
|
|
54
|
* If you want to format all your code inside `src` <kbd>F1 >> Tasks: Run Task >> Format</kbd>
|
|
55
|
* If you want to compile your `sass` files just open file and then hit <kbd>F1 >> Compile - Easy Compile </kbd>.
|
|
56
|
* If you want to also minify hit <kbd>F1 >> Minify Easy Compile</kbd>.
|
|
57
|
* Your code should be formated while you are writing
|
|
58
|
* Everything is set 🏆🎉
|
|
59
|
### Crawler (`./modules/crawler`)
|
|
60
|
To make changes inside crawler module click on file `python.code-workspace` which is in `./modules/crawler` folder
|
|
61
|
* Click on <kbd>Open Workspace</kbd> at the bottom of the screen, wait
|
|
62
|
* Click on <kbd>Reopen in Container</kbd>
|
|
63
|
* Now you are inside the **docker container**. You should see something like this in the bottom left corner
|
|
64
|
|
|
65
|
</br>
|
|
66
|
</br>
|
|
67
|
<img src="./doc/images/dev-container.png">
|
|
68
|
|
|
69
|
* To **close attach or reopen project in container** use these arrows
|
|
70
|
* Now **install recommended extension.**
|
|
71
|
* If you need to run `pip` or `python` commands simply open vscode terminal and run your commands there.
|
|
72
|
* Everything is set 🏆🎉
|
|
73
|
|
|
74
|
### Debuging
|
|
75
|
* Set your breakpoints and hit <kbd>F5</kbd> or goto <kbd>Run >> Start debugging</kbd>
|
|
76
|
* Working in both modules
|
|
77
|
## 🚀 Production env
|
|
78
|
* To run production env on your local machine execute this `./scripts/build.sh`
|
|
79
|
|
|
80
|
<br><hr><br>
|
|
81
|
|
|
82
|
## Add new dataset to heatmap
|
|
83
|
|
|
84
|
Follow these steps:
|
|
85
|
|
|
86
|
1. Run script `modules/crawler/prepare_new_dataset.py` in directory `modules/crawler`.
|
|
87
|
2. Fill items like `display-name`, `display-color`, `dataset-name`, `url`, `regex`, `update-period` within the YAML config file in directory `modules/crawler/DatasetConfigs`.
|
|
88
|
3. Implement own dataset crawler in directory `modules/crawler/DatasetCrawler`.
|
|
89
|
4. Implement own dataset processor in directory `modules/crawler/DatasetProcessing`.
|
|
90
|
5. Run script `modules/crawler/force_update_dataset.py` and enter name of new dataset. After that all needed data are downloaded and processed.
|
|
91
|
6. Fill coordinates to `devices` items in the YAML config file.
|
|
92
|
7. After running `force_update_datasets.py` again, data will be loaded to database.
|
|
93
|
|
|
94
|
<br><hr><br>
|
|
95
|
|
|
96
|
## Auto updating datasets – Cron script
|
|
97
|
|
|
98
|
File `modules/crawler/crone_update_script.py` performs update of all added datasets. Update period is defined in the YAML config file of each dataset (item `update-period`).
|
|
99
|
|
|
100
|
Script `modules/crawler/crone_update_script.py` is executed by systems cron (`modules/crawler/crontab`) every day.
|
|
101
|
|
|
102
|
<br><hr><br>
|
|
103
|
|
|
104
|
## Customize appearance of the heatmap
|
|
105
|
|
|
106
|
### SASS style file
|
|
107
|
CSS color settings (backgrounds, controls, ...).
|
|
108
|
|
|
109
|
`public/css/style.scss`
|
|
110
|
|
|
111
|
### Heatmap locations list
|
|
112
|
Locations with coordinates.
|
|
113
|
|
|
114
|
`public/templates/heatmap-locations.html.twig`
|
|
115
|
|
|
116
|
<img src="./doc/images/heatmap-locations-list.png" style="width: 250px; height: auto;">
|
|
117
|
|
|
118
|
### Intro heatmap webpage
|
|
119
|
`public/templates/index.html.twig`
|
|
120
|
|
|
121
|
### Heatmap webpage
|
|
122
|
`public/templates/heatmap.html.twig`
|