A serios approach to a task queue

Estimating tasks is hard. Essentially, it is predicting the future, which is known to be hard. But we can apply some tricks from queue & resource management in IT systems to personal throughput. All my life I have run from that — maybe now it’s time to take on that problem.

Problems with planning

Lack of data

At the time of estimation the worker does not know:

the amount of time available for him for this project
how long the task will take in reality (discovering undiscovered real-life requirements may dramatically increase the hours)
how much time in the following days will be consumed by tasks and calls yet to be allocated

Humane productivity is not constant

Fragmented time blocks lowers productivity: having one 4-hr time slot is different than having 8 × 30-minute time slots.
Working on several projects in parallel impairs productivity due to forgotten project context.
Sick, having life problems, not enough sleep, weather-dependent etc.

Over-book and over-promise on parallel projects

Changes in tasks are blind to parallel projects: when a manager of one project insists on boosting priority of a task, managers of the other projects don’t get notified and cannot object.

As at the time of estimation, a worker in lacks all the data, nothing stops him from over-promising.

Manual change propagation is expensive

When the worker discovers an imminent delay, all the subsequent tasks need ETA correction, which is crazy expensive in terms of time and attention, thus never happens.

Also, to properly adjust some of the task, managers need to coordinate with that worker, which takes even more time off his calendar.

Undiscovered requirements

Everyone knows it: you start an obviously small task, and discover that some of your assumptions about the underlying system were off. What usually is easy to change, cannot be changed in reasonable amout of time. Something you considered sufficient, is unsufficient, and it will take a lot of time to get it right.

Cascading dependencies

Adjusting ETA of a task will result in a cascade of changes in all tasks planned after it. In a good scenario, one of the future tasks may be thrown off to a distant future. But sometimes all your time will be allocated to high-priority tasks. Adjusting ETA of that task will result in a cascade of changes in all tasks planned after it… ad infinitum.

Circular dependencies

Adjusting ETA of a task may by cascade result in a change of another task, which will affect requirements of the former task. (“if we cannot finish it by the Cyber-monday, then we rather postpone this subproject to the next year and do something else instead, so the former task is not needed”). Better examples to be added

Problems with emotions

Colleagues and clients tend to treat time estimations as promises

Strictly speaking, a promise to accomplish a task by some time is always void, as major discovered dependencies may pop-up at any stage. You can legitimately only promise to do some amount of work. But promising to finish something is a gambling bet. Managers and clients pretend to not understand that and shame the worker to stimulate him.

Shame cripples productivity

A worker who does not deliver on time tends to feel shame, which stimulates, but at the same time adds mental load and decreases cognitive capacity. Slacking on several deadlines can drive a worker to a halt in a pit of anxiousness and shame.

Communicating changes in your ETA too often may even be risky

An employee (or an agency) who is constantly telling you about deadlines creep may leave an impression of being more problematic, while in reality being more honest, transparent and precise than his siblings who juggle all this somehow within, and break deadlines just once in a while (and who doesn’t).

Approach

There is only 1 “consumer” (worker). There are many “producers” (managers, coordinators, colleagues, clients). There are one or more producers per project.

Ideally, when a task expected ready time is changed, all the subsequent tasks’ time must be updated by cascade.

Some tasks have fixed deadlines. Other have “tension” increase with time. Other can be postponed indefinitely.

We basically need a queue storage with constraints and triggers. Constraints like “task with a fixed deadline should not be moved”, triggers like “when a task starts taking more space, push future tasks further and inform related managers“.

What should be done when accepting a new time-sensitive task

ask if the deadline is hard, how much it is important
save in the tasks queue store
guess how many hours it would take
allocate those hours in the available calendar slots
inform the manager when to expect it after allocating in calendar

What should be done when some work has been done

log time in RedMine
update progress in RedMine
update progress in task queue store
guess how much more the task will take to finish
if that guess is more than the remaining time, proceed to the ETA change protocol

What should be done when a task’s ETA changes

Ideally, when a task expected ready time is changed, all the subsequent tasks’ time must be updated by cascade.

For each future task,

skip if it has fixed time (calls etc.)
skip if it has a hard deadline
move forward to the next available time slot
if after moving the new task ETA breaks an agreement or a promise of some sort, that task should either be prioritized above the tasks with no such agreement and moved back
if all back is occupied with tasks with more priority, the manager should be informed

For that, when being moved, each time slot of a task in the calendar needs to be looked-up for the metadata somewhere. The RedMine issue record has no built-in deadline column, so we might as well have a separate Numbers spreadsheet.

Conflicting deadlines

After correcting ETA on a task, several tasks with hard deadlines might end in a conflict. What should be done?

Summon a discussion with the managers of the tasks en causa to decide either one:

Delegate to a colleague
scale workforce by hiring a freelancer
push one of the tasks forward and inform the client.

What about “Critical chain” / “Critical path”?

Ain’t I solving the problem that’s been solved? The project is essentially a task graph.

Problems with execution

An employee has no clear queue of tasks. Basically, to always work on the right task, the employee needs a list of tasks with descriptions and time limit per each. Instead, he has a cloud of directions (oral, in Slack, in email, in Trello etc.) about how to resolve the priority of tasks. In other words, there is no system removing from an employee the effort of deciding what to do.

Instruments

Let’s compare some popular approaches to represent the tasks.

Type of representation	ToDo list		Calendar	Kanban board		Gannt chart
How many projects visible at once	1 project	multi-project	multi-project	1 project	multi-project	1 project	multi-project
How many people visible at once	me	team	me	team	me	team	me
Shows how long a person should have been worked on a task according to plan, by now	❌	❌	✅	❌	❌	✅	✅
Allows changeing the order of future tasks	✅	✅	✅	✅	✅	✅	✅
Shows dependencies between tasks	❌	❌	❌	❌	❌	✅	✅
Warns when rescheduling a task forward breaks constraints of that task	❌	❌	❌	❌	❌	✅	✅
Warns when rescheduling a task forward breaks constraints of a dependent task	❌	❌	❌	❌	❌	✅	✅
Warns when a woker has more hours per day booked than available	❌	❌	✅	❌	❌	❌	❌

As we see, task management systems are good at creating and changing tasks, but are very bad at preserving constraints such as overbooking of workers within a day, violating deadlines etc.

And it’s understandable: rescheduling a task causes a cascading wave of changes in a lot of future tasks — some of which depend on the results of the original task, the others depend on the availability of the worker.