I have some data in csv format that I need to load into the application once and then reuse it throughout the lifetime of the application, which is across multiple requests. How can I do that?

An obvious method for me is to have a module that will load the data and then expose it. However, I don't like modules doing a lot of work, because then imports lead to unexpected side effects. I would like to do that one-time work in a predictable, deterministic fashion, not because someone imported a module.

Does Django provide some hooks for globals? Some kind of Application/Service class where I could do that work and then access the data in the requests?

I have some data in csv format that I need to load into the application once and then reuse it throughout the lifetime of the application, which is across multiple requests. How can I do that?

An obvious method for me is to have a module that will load the data and then expose it. However, I don't like modules doing a lot of work, because then imports lead to unexpected side effects. I would like to do that one-time work in a predictable, deterministic fashion, not because someone imported a module.

Does Django provide some hooks for globals? Some kind of Application/Service class where I could do that work and then access the data in the requests?

• python
• django

Share Improve this question Follow asked Nov 21, 2024 at 5:16 gruszczygruszczy 42.2k3131 gold badges135135 silver badges186186 bronze badges 3

• settings.py seems like the perfect place to load that kind of data. – John Gordon Commented Nov 21, 2024 at 5:17
• 2 You can also simply use functools.cache to cache the function that returns the data indefinitely. Note that it will be cached per process, so if you are spinning up 4 workers (using gunicorn etc) you will cache 4 copies of the same data. If the data is too large consider using a separate service such as redis or memcached with Django's cache framework. – Selcuk Commented Nov 21, 2024 at 5:20
• A module can still hold the data, even if it doesn't automatically load it when you import. Modules are automatically cached so it's only initialized once. – Mark Ransom Commented Nov 21, 2024 at 13:27

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

You can use AppConfig to initialize the data : (See https://docs.djangoproject/en/5.1/ref/applications/)

class MyConfig(AppConfig):
    name = 'my_app'

    def ready(self):
        self.my_data = ...  # Read data here

You can retrieve the data in the code of your views:

from django.apps import apps

def my_view(request, ...):
    my_var = apps.get_app_config('my_app').my_data

But in my opinion this is unnecessarily complex. You better do this in a module, as you yourself indicate:

# Note: the code below is not in a function or a class, but directly in the module!
# It will be executed only once when the server starts and the file is imported,
# `my_data` will continue to exist throughout server lifetime.

my_data = ... # Read data here.

def my_view(request, ...):
    my_var = my_data[...]  # You can use the data here 

And you can import the data in any other module, I don't see why it would lead to "unexpected side effects", as long as you make sure you don't introduce circular imports:

from my_module import my_data

def my_view(request, ...):
    my_var = my_data[...]  # You can use the data here 

This way, my_data will be initialized in a "predictable, deterministic fashion".