Optimization models
The choice of which problem to solve (the meta-problem) can be written as an optimization problem. If you are not already familiar with math models like linear programming, you might want to refer to the meta-problem cycle page instead.
Below are both a deterministic formulation of the meta-problem and some changes to make to develop a stochastic model.
Deterministic meta-problems
Definitions
p is a completely defined problem. Which means all solutions to p have the same value.
b is an incompletely defined problem (a problem space), which means because of undefined parts of the problem (goals and / or constraints), we need to make assumptions to arrive at a solvable completely defined problem. Different choices of assumptions may lead to solutions that have a different value to stakeholders.
e is the effort it takes to solve a given problem with a particular method. Different methods to solve the same problem can take different levels of effort.
Optimization model
The deterministic meta-problem when given an incompletely defined problem b is the choice of which completely defined problem p you select out of the problem-space.
The objective function is to maximize the multi-dimensional value of solving the chosen problem p less the multi-dimensional effort of solving it.
Insights
1. Since we balance effort and value, lower effort can offset higher value. For example, even if it is less accurate, if you can write your problem in a way that is easier to solve it may be worth it.
2. Solving one problem might not make it easier to solve some other related problem. This is true even if they're both in the problem space of p. For example, a minimum viable product may have major gaps compared to how you would develop a piece of software for the long term.
3. Depending on which problem p you choose in the problem-space, different methods might be appropriate. A linear program can be solved with simplex while an integer program might use branch-and-bound.
4. If we have settled on a completely defined problem p in the problem-space of b, solving the meta-problem means choosing the lowest effort method to solve it.
5. If the effort to solve a problem is higher than the value, the best choice is to do nothing.
6. Given an incompletely defined problem b, there are a set of possible optimal solutions that correspond to different choices of assumptions within the problem-space.
Practical usage
When applying the meta-problem approach, we don't typically create an actual optimization math program over problem space. Instead, we use it as a framework to explore the neighborhood in problem-space of an initial completely defined problem p. Our goal is to identify any problems in problem-space which are better than our starting point.
As a result of the meta-problem framework, we also have an approach to develop a list of criteria that would make an alternative problem better to solve. This would include higher profit, benefits received sooner, more ethical solutions, less environmental impact, lower effort to solve, and any other criteria we might consider putting into the objective function or constraints of the meta-problem.
One extension of this mindset is we can also evaluate the implied solution-space instead of only focusing on the problem-space.
Stochastic meta-problems
Changes for problems with uncertainty
Formally defining a stochastic meta-problem is tricky as we have the same complexity as in the deterministic case – but now have to consider the unknown as well.
Since the meta-problem is solved before solving the actual problem, there may be some uncertainty that will be realized outside the scope of our analysis. However, some uncertainty could be realized as we are solving the meta-problem.
As a result of this shift, our focus for stochastic meta-problems is to identify a series of subproblems which will converge to the optimal problem to solve.
The objective function will be to maximize the sum of the multi-dimensional value of solving the chosen problems over the series, less the sum of the multi-dimensional effort of solving them.
Practical Usage
When applying the meta-problem approach, the stochastic formulation explicitly accounts for the discovery process that happens as we explore the problem and solution spaces.
As an example, we often don’t learn all our criteria for a decision problem until we explore the problem-space. Said another way, it’s when you present a recommendation that decision makers often highlight other criteria that they care about, and the proposed solution does not satisfy, because they were late-arriving requirements.
Knowing that there is uncertainty which can be resolved depending on which problem you solve in the series, we can be strategic about which problems we solve early on.
Additionally, sometimes our biggest uncertainty has to do with the effort it will take to solve a problem.
This insight explains project management methodologies like Agile which ask the team to re-assess at regular intervals whether or not the problem is “solved.”