I am building a Meteor application that is using a mongo database.

I have a collection that could potentially have 1000s of documents that need to be updated at different times.

Do I run setTimeouts on creation or a cron job that runs every second and loops over every document?

What are the pros and cons of doing each?

To put this into context:

I am building an online tournament system. I can have 100s of tournaments running which means I could have 1000s of matches.

Each match needs to absolutely end at a specific time, and can end earlier under a condition.

I am building a Meteor application that is using a mongo database.

I have a collection that could potentially have 1000s of documents that need to be updated at different times.

Do I run setTimeouts on creation or a cron job that runs every second and loops over every document?

What are the pros and cons of doing each?

To put this into context:

I am building an online tournament system. I can have 100s of tournaments running which means I could have 1000s of matches.

Each match needs to absolutely end at a specific time, and can end earlier under a condition.

• javascript
• node.js
• meteor
• cron
• settimeout

Share Improve this question Follow edited Dec 17, 2020 at 0:34 peterh 1 asked Dec 12, 2014 at 19:41 gbachikgbachik 1,70633 gold badges1717 silver badges3030 bronze badges

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 4

Using an OS-level cron job won't work because you can only check with a 60-second resolution. So by "cron job", I think you mean a single setTimeout (or synced-cron). Here are some thoughts:

Single setTimeout

strategy: Every second wake up and check a large number of matches, updating those which are plete. If you have multiple servers, you can prevent all but one of them from doing the check via synced-cron.

The advantage of this strategy is that it's straightforward to implement. The disadvantages are:

1. You may end up doing a lot of unnecessary database reads.
2. You have to be extremely careful that your processing time does not exceed the length of the period between checks (one second).

I'd remend this strategy if you are confident that the runtime can be controlled. For example, if you can index your matches on an endTime so only a few matches need to be checked in each cycle.

Multiple setTimeouts

strategy: Add a setTimeout for each match on creation or when the sever starts. As each timeout expires, update the corresponding match.

The advantage of this strategy is that it potentially removes a considerable amount of unnecessary database traffic. The disadvantages are:

1. It may be a little more tricky to implement. E.g. you have to consider what happens on a server restart.
2. The naive implementation doesn't scale past a single server (see 1).

I'd remend this strategy if you think you will use a single server for the foreseeable future.

---

Those are the trade-offs which occurred to me given the choices you presented. A more robust solution would probably involve technology outside of the meteor/mongo stack. For example, storing match times in redis and then listening for keyspace notifications.

This is all a matter of preference, to be honest with you.

I'm a big fan of writing small, independent programs, that each do one thing, and do it well. If you're also like this, it's probably better to write separate programs to run periodically via cron.

This way you get guaranteed OS-controlled precision for the time, and small, simple programs that are easy to debug outside the context of your webapp.

This is just a preference though.