Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed all incorrect references to celery #8887

Merged
merged 1 commit into from
Jul 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions doc/adv-manual/general.rst
Original file line number Diff line number Diff line change
Expand Up @@ -944,9 +944,9 @@ existing code.
Another tricky point in cluster situations is that we need
to transfer gigabytes of data, i.e. the large arrays coming from
scientific calculations, rather than lots of small
messages. Hence, since recent versions of the engine, we are returning
messages. Hence, we are returning
data from the tasks via zeromq instead of using celery/rabbitmq as we
did in the past. This
did in the remote past. This
is hidden from the user, but internally the engine keeps track of all
tasks that were submitted and waits until they send the message that
they finished. If one task runs out of memory badly and never sends
Expand Down
4 changes: 1 addition & 3 deletions doc/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,7 @@

* **worker process**

A process which is dedicated to task execution. Worker process are simply celeryd processes which are configured to execute OpenQuake Engine task code.

For more information about Celery, see http://celeryproject.org/.
A process which is dedicated to task execution.

* **worker machine**

Expand Down
2 changes: 1 addition & 1 deletion doc/hardware-suggestions.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Our cluster master node "wilson" is a Dell® PowerEdge™ R720 with 16 Xeon core

For our largest calculations on a continental or global scale we use a cluster composed of "wilson" (see above) acting as a "master" and two clusters composed by 5 worker nodes (Dell® PowerEdge™ M915 blades) each with 4x 16 cores AMD® Opteron™ and 128GB of RAM for the first and 4 Dell® PowerEdge™ M630 blades (2x 20t/10c Intel® Xeon™ E5-2640v4, 128GB RAM each) for the second cluster. Worker nodes do not need much disk since the data is stored in the master filesystem.

Network is made with a link aggregation between two 10 gigabit connections; however, up to a couple hundred celery workers a single gigabit connection is enough.
Network is made with a link aggregation between two 10 gigabit connections; however, up to a couple hundred workers a single gigabit connection is enough.

Windows is not supported for large scale deployments.

Expand Down
2 changes: 0 additions & 2 deletions doc/installing/ubuntu.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@ sudo apt install python3-oq-engine
The package installs three system service managed through [systemd](https://www.freedesktop.org/software/systemd/man/systemctl.html):
- `openquake-dbserver`: provides the database for the OpenQuake Engine and must be started before running any `oq engine` command
- `openquake-webui`: provides the WebUI and is optional
- `openquake-celery`: used only on a multi-node setup, not used in a default setup

`openquake-dbserver` and `openquake-webui` are started by default at boot.

To manually start, stop or restart a service run
Expand Down
4 changes: 2 additions & 2 deletions doc/sphinx/baselib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ openquake.baselib
=================

`openquake.baselib` is a collection of utilities to run large numeric
simulations. It includes support for parallel computing (via multiprocessing,
celery, zmq or dask), performance monitoring and a serialization library
simulations. It includes support for parallel computing (via multiprocessing
or zmq), performance monitoring and a serialization library
from Python objects to HDF5 files and viceversa.

Development and support
Expand Down
5 changes: 0 additions & 5 deletions openquake/baselib/tests/parallel_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,6 @@
import numpy
from openquake.baselib import parallel, general, hdf5, performance

try:
import celery
except ImportError:
celery = None


def get_length(data, monitor):
return {'n': len(data)}
Expand Down
74 changes: 0 additions & 74 deletions openquake/engine/celeryconfig.py

This file was deleted.

1 change: 0 additions & 1 deletion utils/reset-db
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ echo ''
echo "!! ${BOLD}This script is going to remove the OpenQuake Engine data${NORMAL}"
echo '!!'
echo -e "!! \t* Make sure no calculations are running"
echo -e "!! \t* Make sure Celery is stopped, if used"
echo -e "!! \t* The OpenQuake Engine datastores located in /home/[user]/oqdata will be removed"
echo -e "!! \t* The OpenQuake Engine database will be dropped"
echo -e "!! \t All the old data contained will be lost"
Expand Down