Retry recreate_index subtasks on worker exit

[mbertrand]

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/5615

Description (What does it do?)

• Enforce cold shutdowns for the celery container because tasks are not retried on warm shutdowns (bug? Celery Tasks cannot be Revoked after issuing Warm Shutdown celery/celery#8417)
• Refactor task options

Screenshots (if appropriate):

• Desktop screenshots
• Mobile width screenshots

How can this be tested?

Open 3 terminals:

• Terminal 1: docker compose up
• Terminal 2: when ready, ./manage.py recreate_index --all
• Terminal 3: Once you start seeing a bunch of recreate_index subtasks running like index_contentfiles, run docker compose stop celery. Wait until you see that celery has exited in terminal 1. Then run docker compose up celery
• Once celery has restarted, the recreate_index tasks should resume, and eventually the task in terminal 2 should complete successfully.

Additional Context