|
1 年之前 | |
---|---|---|
config/db | 1 年之前 | |
spark-packages | 2 年之前 | |
src/spark | 1 年之前 | |
.gitignore | 1 年之前 | |
README.md | 2 年之前 | |
bench.py | 1 年之前 | |
clean.py | 2 年之前 | |
settings.json | 2 年之前 | |
setup.py | 1 年之前 | |
small_test_data.csv | 2 年之前 | |
start_services.sh | 2 年之前 | |
submit.sh | 2 年之前 | |
submit_graph.sh | 2 年之前 | |
submit_partition.sh | 2 年之前 |
TODO
For the graph implementation specifically you need to install graphframes
manually from a third party since the official release is incompatible with spark 3.x
(pull request pending). A prebuilt copy is supplied in the spark-packages
directory.
settings.json
to reflect your setup. If you are running everything locally you can use start_services.sh
to turn everything on in one swoop. It might take a few minutes for Cassandra to become available.python3 setup.py
from the project root. Per default this will move small_test_data.csv
into the transactions table.submit.sh
(slow) or submit_graph.sh
(faster)python3 clean.py
. Be wary that this wipes all table definitions and data.