Ask HN: How do you approach a problem you are not sure has a solution?
How do you battle against (self-inflicted) anxiety/paralysis when you are attempting to tackle a problem you are not sure has a solution? I have a very open mind to solving problems but it can make it difficult to come to conclusions. Anyone know what I'm talking about here? Have any advice?
303 comments
One thing that my grad advisor used to emphasize is “failing fast”. That is, for every problem of the form “Is it possible to do X?”, there is a dual problem of “Can you prove it’s not possible to do X?”
Before spending far too much time on the first question, it’s worth it to spend a little bit of time on the second: what’s the quickest way to show that this can’t possibly work? Often this takes the form of looking for statements such as “If X worked, then Y would work too”, and then you go test Y. Just because Y holds, doesn’t mean X does… but if it doesn’t, you know X doesn’t.
It can feel like a bit of a diversion in the moment (“why am I wasting my time with Y, when I really care about X?”) but it has saved me months, possibly years of going down rabbit holes in my career. Likewise, it definitely helps with that anxiety you mention, because it means I at least have some motivation that my idea isn’t completely crazy.
Before spending far too much time on the first question, it’s worth it to spend a little bit of time on the second: what’s the quickest way to show that this can’t possibly work? Often this takes the form of looking for statements such as “If X worked, then Y would work too”, and then you go test Y. Just because Y holds, doesn’t mean X does… but if it doesn’t, you know X doesn’t.
It can feel like a bit of a diversion in the moment (“why am I wasting my time with Y, when I really care about X?”) but it has saved me months, possibly years of going down rabbit holes in my career. Likewise, it definitely helps with that anxiety you mention, because it means I at least have some motivation that my idea isn’t completely crazy.
Love this comment because it translates so well to any mentally taxing endeavor. Writing a long-running program? Set a few assertions up front so that it fails immediately before wasting your time. Think about those "failure" condition assertions up front and save your time on practically any experiment. Even chess players do this by surrendering early when they know a game has reached a conclusion.
> Writing a long-running program?
If it applies, have a "testing" version that runs quickly, on limited (but error-prone) data cases. Ideally, run it with the rest of your test suite.
If it applies, have a "testing" version that runs quickly, on limited (but error-prone) data cases. Ideally, run it with the rest of your test suite.
+1...but this may be better advice when you don't have any anxieties/paralysis about the problem than when you do. Turned-out-that-it-was-overconfident approaches to problems have probably wasted far more time than anxieties/paralysis has.
Peripheral to this, some of my best refactors have come from trying to document why we can’t do X, explaining that we can’t do X because of Y, and realizing that Y does not need to exist.
There comes a point where explaining why we can’t have something is more painful than just fixing it.
There comes a point where explaining why we can’t have something is more painful than just fixing it.
This is also a useful technique to validate a new product before committing too many resources building it. Imagine that you have built it technically flawlessly but it's not selling copies. Why might that be? Draw up a list of reasons and test each of those as hypotheses before you build the product itself.
Something similar we do at work is write a quick document on why something isn't possible.
A bug can't be reproduced? Take a video of it working fine, and show the logical path for how the bug couldn't happen.
Can't build X in 3 months? What's the minimum for X? What's the missing component that takes 2 months? We can't have a partnership with A, but why wouldn't it be possible to have a partnership with A's competitor?
A bug can't be reproduced? Take a video of it working fine, and show the logical path for how the bug couldn't happen.
Can't build X in 3 months? What's the minimum for X? What's the missing component that takes 2 months? We can't have a partnership with A, but why wouldn't it be possible to have a partnership with A's competitor?
A risk to this approach: if you (or someone working for you, who is trying to tackle a problem) starts to externalize the motivation to solve the problem, optimization may start to favor 'failing fast' rather than pursuing best approaches (e.g. there's only one chance to try something per day, and you/they develop a pattern of pursuing the simplest possibility because it takes the least research and thus makes for the easiest increment of work day.)
This is possibly a risk, but it's not one I've ever seen play out. I don't think most people favor failing fast. It's something you do because it's good practice, while crossing your fingers that it doesn't fail. I think this is because in order to prove that something doesn't work, you first have to come up with that something -- deciding on a problem and thinking of a potential solution. While some people might find it fun to disprove other people's ideas, I don't think most really find it satisfying to disprove their own.
I think the bigger risk to this approach is that it can be disheartening and demotivating to see your ideas that you already got attached to and were looking forward to pursuing shut down. But it's still less demotivating than spending a year on it before watching it fail.
I think the bigger risk to this approach is that it can be disheartening and demotivating to see your ideas that you already got attached to and were looking forward to pursuing shut down. But it's still less demotivating than spending a year on it before watching it fail.
I had a group working under me who were troubleshooting an intermittent fault in some control circruitry for a nuclear reactor, and they started phoning it in just like this when they realized that there was only enough appetite to describe and propose weird problems to the boss once each day. After about a week, the goal became achievement of the daily failure with a minimum of preparation. It was a highly solvable problem, but they were all avoiding the necessary step of serious research.
That's reasonable. I think being in the academic world, I don't see it as often because daily achievement isn't really rewarded -- it's all about individual output in the longer term (manuscripts, grants, etc. ), and every "failure" feels like a step away from having a completed manuscript, even when it's necessary. If anything, it's why a lot of academics tend to avoid the "fail fast" techniques, because at a daily level, it feels more productive to be working towards a solution (even if it is a waste of time) than shooting down potential solution after solution.
Can you share a story about this?
Here's an example from my own work:
I had a function that performed a 256-bit division. I thought there was a good chance it could be optimized, because the dividend was fixed: the function computed 2^256 / d. Surely, if the dividend was always the same (and such a nice, power-of-2 number!), there should be a way to exploit that, right?
I poked at it for a few hours before it occurred to me that I could ask someone else who knew a lot about division. So I cold-emailed Daniel Lemire. He replied with the following argument:
Suppose you had an algorithm that could compute 2^k / d much more efficiently than the typical algorithm. You could then use this algorithm to speed up any division n / d, by rewriting the expression as (n - 2^k)/d + 2^k/d. This works because both dividends will now be less than 2^k, and division algorithms get faster as the dividend gets smaller.
In other words: it's not impossible that there's an efficient algorithm for 2^k / d, but if it does exist, then all the people who have dedicated their time to optimizing integer division over the years have missed some enormous low-hanging fruit.
That was compelling enough for me, and I gave up trying to optimize my function. :)
I had a function that performed a 256-bit division. I thought there was a good chance it could be optimized, because the dividend was fixed: the function computed 2^256 / d. Surely, if the dividend was always the same (and such a nice, power-of-2 number!), there should be a way to exploit that, right?
I poked at it for a few hours before it occurred to me that I could ask someone else who knew a lot about division. So I cold-emailed Daniel Lemire. He replied with the following argument:
Suppose you had an algorithm that could compute 2^k / d much more efficiently than the typical algorithm. You could then use this algorithm to speed up any division n / d, by rewriting the expression as (n - 2^k)/d + 2^k/d. This works because both dividends will now be less than 2^k, and division algorithms get faster as the dividend gets smaller.
In other words: it's not impossible that there's an efficient algorithm for 2^k / d, but if it does exist, then all the people who have dedicated their time to optimizing integer division over the years have missed some enormous low-hanging fruit.
That was compelling enough for me, and I gave up trying to optimize my function. :)
Feels very much like an appeal to authority though. Fast inverse square root was discovered/implemented as a side task for a first person shooter: https://en.wikipedia.org/wiki/Fast_inverse_square_root.
Appeal to authority is only a fallacy when the person is an authority in an unrelated area. Appealing to an actual authority of the subject in question isn't a fallacy—it's the whole point of having authorities.
"90% of dentists agree you should own a motorbike" is an appeal to authority fallacy.
"90% of dentists agree you should floss" is not.
"90% of dentists agree you should own a motorbike" is an appeal to authority fallacy.
"90% of dentists agree you should floss" is not.
You are mistaken.
An appeal to authority is any logical argument that relies significantly upon the authority of the person making that argument as support for that argument. It then becomes circular and flawed.
If you were to replace the person who is actually speaking with anyone else, and look at the same argument and its support, and it doesn't hold up, that's an appeal to authority.
It doesn't matter who the authority is. Just that there was an authority used to support a significantly unsupported argument
An appeal to authority is any logical argument that relies significantly upon the authority of the person making that argument as support for that argument. It then becomes circular and flawed.
If you were to replace the person who is actually speaking with anyone else, and look at the same argument and its support, and it doesn't hold up, that's an appeal to authority.
It doesn't matter who the authority is. Just that there was an authority used to support a significantly unsupported argument
You are mistaken, and I will argue my claim by deferring to Wikipedia, which is an authority on fallacies:
https://en.wikipedia.org/wiki/Argument_from_authority
https://en.wikipedia.org/wiki/Argument_from_authority
First sentence:
> An argument from authority (argumentum ab auctoritate), also called an appeal to authority, or argumentum ad verecundiam, is a form of argument in which a claim made by an authority on some topic is used as evidence to support one's own claim.
Further down:
> Historically, opinion on the appeal to authority has been divided: some hold that it can be a valid or at least defeasible.
You seem to be arguing for appeals to authority as defeasible.
Incorrectly understanding the authority you are quoting shows how problematic it is to make pure appeals to authority.
> An argument from authority (argumentum ab auctoritate), also called an appeal to authority, or argumentum ad verecundiam, is a form of argument in which a claim made by an authority on some topic is used as evidence to support one's own claim.
Further down:
> Historically, opinion on the appeal to authority has been divided: some hold that it can be a valid or at least defeasible.
You seem to be arguing for appeals to authority as defeasible.
Incorrectly understanding the authority you are quoting shows how problematic it is to make pure appeals to authority.
I read GP as a joke.
I don't think that's the definition of the fallacy most people have.
argumentum ad populum much?
Lol. But seriously appeal to authority fallacy need not be only when the authority is outside their domain. For instance when authority makes a bad argument.
The dividing line here is that it's fallacious to argue that something is true simply based on authority. That doesn't hold where there are external cogent reasons for following that authority (eg. in this example: it's a difficult problem that's well studied by many, and the body of work has not yet yeileded an answer such that it's unlikely that a more casual hunch will do so).
I'm not saying that this type of proof was applied perfectly here, however, the notion of proving that solving problem X must be hard because solving it would solve problem Y, which is (at least for now) known to be hard is a fundamental methodology in the field of computational complexity theory. There is a strong academic basis in these types of lines of reasoning, specifically in computing and optimization.
I'm not a trained computer scientist but didn't Daniel Lemire (purportedly) answer a different question from what OP asked? In OPs question k is fixed, wouldn't that open up avenues to implementation different from when k is not?
More like noting that many people before have tried and failed.
I can see why you puzzled over this. I’m not sure I agree with the guys argument — seems more like that there isn’t asymptotically better algorithm. It’s not convincing to me that there isn’t one division algorithm that’s a constant factor better.
IIRC you are correct, there are one or two spots in the Knuth division algorithm where having a power-of-2 dividend lets you eliminate a shift or an addition or something -- but nothing (nothing obvious, at least!) that yields a significant speedup.
[deleted]
One thing I like to ask myself is how did things get to be the way they are?
>all the people who have dedicated their time to optimizing . . . over the years have missed some enormous low-hanging fruit.
When I was a teenager one of my most worthwhile early chemical breakthroughs as an undergraduate was a typical lesson (these are dubbed "experiments") where students produced a product from raw materials and were evaluated on their final yield of acceptable recovered product, compared to the theoretical amount that could be obtained if perfection were to be achieved across all steps in the reaction and subsequent processing & handling.
This was a reaction that was known not to go very far to completion when equilibrium was reached to begin with, so most emphasis was placed on technique for subsequent separation and purification of the desired product recovered afterward. Good yield was considered 25%, poor at 10% or less, the historical high score was 27%. It was also accepted that the final product was not considered very stable, subject to degradation from exposure to things like heat, air and moisture, and there were some literature references this was based on.
Basically an overall view of the combined laboratory techniques applied across the study, compared to how the other students were doing. The product was not actually a commercially useful material but it had proven worthwhile in this regard.
The university had been doing this same challenging competition for decades, designed back then by a still-active professor, and it was considered a good comparison of how each year's students were performing on the same real-world problems over the decades. Same old same old.
I went through it one time and it came out pretty good, but before refining my technique I dived a bit deeper into the physical properties of all chemicals involved, in my case looking for a way to drive the equilibrium further to begin with.
One of the well-known ways to drive reversible reactions to completion is to remove product as you go along, not always easy but also quite essential in many industrial processes.
Seemed to me distillation would be most feasible except the product was a solid and one of the raw materials was a liquid having a known boiling point, much lower than the expected boiling point of the dissolved solid raw material product in the reaction flask, and the solvent much lower in boiling point than that.
Ended up vacuum distilling from the incompleted reaction flask where the water of reaction was removed along with the solvent, "excessive" heat actually helped complete the reaction before the remaining lower-boiling raw material was vaporized, and the desired solid product had a lower boiling point under vacuum while melted than the remaining "heavy solid" raw material, and the good stuff was recovered in over 50% yield as an oil in it's own dedicated receiver, which crystallized wonderfully by itself with no further purification needed. Earned me my first A+ and encouraged me to continue going further than average ever since.
Turns out all the original professors were wrong about heat instability, and also had never fully considered as many physical properties of the exact chemicals being worked with, only similar materials for which there was much more common knowledge and published references.
The final lesson was that sometimes the most respected elements of "common sense" amount to more or less "common lack of sense".
Also, I've said this before, when all recommended solutions have been attempted and failed to deliver, the actual solution will be something that is not recommended.
>all the people who have dedicated their time to optimizing . . . over the years have missed some enormous low-hanging fruit.
When I was a teenager one of my most worthwhile early chemical breakthroughs as an undergraduate was a typical lesson (these are dubbed "experiments") where students produced a product from raw materials and were evaluated on their final yield of acceptable recovered product, compared to the theoretical amount that could be obtained if perfection were to be achieved across all steps in the reaction and subsequent processing & handling.
This was a reaction that was known not to go very far to completion when equilibrium was reached to begin with, so most emphasis was placed on technique for subsequent separation and purification of the desired product recovered afterward. Good yield was considered 25%, poor at 10% or less, the historical high score was 27%. It was also accepted that the final product was not considered very stable, subject to degradation from exposure to things like heat, air and moisture, and there were some literature references this was based on.
Basically an overall view of the combined laboratory techniques applied across the study, compared to how the other students were doing. The product was not actually a commercially useful material but it had proven worthwhile in this regard.
The university had been doing this same challenging competition for decades, designed back then by a still-active professor, and it was considered a good comparison of how each year's students were performing on the same real-world problems over the decades. Same old same old.
I went through it one time and it came out pretty good, but before refining my technique I dived a bit deeper into the physical properties of all chemicals involved, in my case looking for a way to drive the equilibrium further to begin with.
One of the well-known ways to drive reversible reactions to completion is to remove product as you go along, not always easy but also quite essential in many industrial processes.
Seemed to me distillation would be most feasible except the product was a solid and one of the raw materials was a liquid having a known boiling point, much lower than the expected boiling point of the dissolved solid raw material product in the reaction flask, and the solvent much lower in boiling point than that.
Ended up vacuum distilling from the incompleted reaction flask where the water of reaction was removed along with the solvent, "excessive" heat actually helped complete the reaction before the remaining lower-boiling raw material was vaporized, and the desired solid product had a lower boiling point under vacuum while melted than the remaining "heavy solid" raw material, and the good stuff was recovered in over 50% yield as an oil in it's own dedicated receiver, which crystallized wonderfully by itself with no further purification needed. Earned me my first A+ and encouraged me to continue going further than average ever since.
Turns out all the original professors were wrong about heat instability, and also had never fully considered as many physical properties of the exact chemicals being worked with, only similar materials for which there was much more common knowledge and published references.
The final lesson was that sometimes the most respected elements of "common sense" amount to more or less "common lack of sense".
Also, I've said this before, when all recommended solutions have been attempted and failed to deliver, the actual solution will be something that is not recommended.
I often approach this from a “upper bound” and “lower bound” scenario perspective. So suppose I am trying to extract a signal from data that is noisy and sparse. Often the most time consuming part of solving the problem is figuring out how to deal with all of the noise, and I don’t want to waste time doing that just to come up with nothing in the end.
A “upper bound” approach would be to simulate some data that is perfectly clean and meets all my model assumptions, and test my basic method on it. If it doesn’t work on the clean data, it won’t work on the noisy data.
The “lower bound” approach would be to try the simplest and dumbest thing I can think of to deal with the noise in the real data. If I can pick up some amount of signal even with the dumb approach, it makes me much more confident that if I spend time making a sophisticated noise model, it will be worth it.
A “upper bound” approach would be to simulate some data that is perfectly clean and meets all my model assumptions, and test my basic method on it. If it doesn’t work on the clean data, it won’t work on the noisy data.
The “lower bound” approach would be to try the simplest and dumbest thing I can think of to deal with the noise in the real data. If I can pick up some amount of signal even with the dumb approach, it makes me much more confident that if I spend time making a sophisticated noise model, it will be worth it.
Probably not quite what you wanted, but P=NP. When you look at what does that imply then it's hard to think that it holds.
Scott Aaronson has a checklist on how to quicky reject a P?=NP paper, and one of the best methods is to check whether the paper proves something weaker first.
Scott Aaronson has a checklist on how to quicky reject a P?=NP paper, and one of the best methods is to check whether the paper proves something weaker first.
Back off the envelope calculations can usually quickly tell you if something is possible or not.
For physical problems. Newtons law of conservation of energy is an obvious first sanity check. Given some objects mass and the desired travel, you can get the required energy and from there the required power. Before you even start designing a prototype.
For physical problems. Newtons law of conservation of energy is an obvious first sanity check. Given some objects mass and the desired travel, you can get the required energy and from there the required power. Before you even start designing a prototype.
I sorry, but I am truly confused at the logic here.
If X, then Y.
To test X works you first you start on Y?
If Y succeeds then you don't know if X failed or not && if Y fails then you know that X failed too?
If X, then Y.
To test X works you first you start on Y?
If Y succeeds then you don't know if X failed or not && if Y fails then you know that X failed too?
In formal logic, it is known as "modus tollens"... if 'if x then y' is true, then 'if not y then not x' is also true. The inverse is not necessarily true, however: 'if x then y' does not mean 'if y then x'.
In the case of a an X that is hard to figure out on its own, and it is easy to figure out Y, then it might be worth testing Y first, even if you will only get useful information if Y is false.
In the case of a an X that is hard to figure out on its own, and it is easy to figure out Y, then it might be worth testing Y first, even if you will only get useful information if Y is false.
Let's say you're trying to figure out whether you left your coat at work. You could drive to the office and look for it there, and that will definitely tell you the answer. But if you look in your closet at home, then you know that the coat is definitely not at work, so you're done.
If you don't find the coat in your closet, you will don't know whether you left it at work. It could be in your car or maybe you left it on the train. But it's still worth checking your closet first because that gives you an early possible solution.
In logic, that's: If coat is at work, then closet is empty.
X = "Coat is at work".
Y = "Closet is empty".
Modus tollens tells us that "If X, then Y" implies "If not Y, then not X." So: If closet is not empty, then coat is not at work.
If you don't find the coat in your closet, you will don't know whether you left it at work. It could be in your car or maybe you left it on the train. But it's still worth checking your closet first because that gives you an early possible solution.
In logic, that's: If coat is at work, then closet is empty.
X = "Coat is at work".
Y = "Closet is empty".
Modus tollens tells us that "If X, then Y" implies "If not Y, then not X." So: If closet is not empty, then coat is not at work.
Well, assume you have a very, very efficient algorithm to check if normal boolean expressions have a solution. It checks some constant number of things for each variable, and then outputs a solution and it works for a large number of things.
Using the same logic as the parent comment, I would be very suspicious of the general applicability of this algorithm. Because, if this algorithm was correct, P would be equal to NP based off of this algorithm, because you'd have a polynomial solution to SAT. This, in turn, would invalidate pretty much all practical cryptography, most likely turn bitcoin on its head, and cause a significant number of other disruptions.
That is this line of thinking. The formal name is Modus Tollens, but it basically says: If your assumption is right, I can propose a much more preposterous assumption that would also be right. Or I could propose something enabled or validated by your assumption which is much easier to invalidate.
I constantly use this in stupid security discussions as well. There are so many people asking about silly threat scenarios, but the specific threat scenarios generally imply that an attacker already has control of critical infrastructure anyway, and all of these nitpicky things they wonder about are just not relevant. Like, if you assume this action to be possible, they have control of the secret management solution, and then we are doomed already.
Using the same logic as the parent comment, I would be very suspicious of the general applicability of this algorithm. Because, if this algorithm was correct, P would be equal to NP based off of this algorithm, because you'd have a polynomial solution to SAT. This, in turn, would invalidate pretty much all practical cryptography, most likely turn bitcoin on its head, and cause a significant number of other disruptions.
That is this line of thinking. The formal name is Modus Tollens, but it basically says: If your assumption is right, I can propose a much more preposterous assumption that would also be right. Or I could propose something enabled or validated by your assumption which is much easier to invalidate.
I constantly use this in stupid security discussions as well. There are so many people asking about silly threat scenarios, but the specific threat scenarios generally imply that an attacker already has control of critical infrastructure anyway, and all of these nitpicky things they wonder about are just not relevant. Like, if you assume this action to be possible, they have control of the secret management solution, and then we are doomed already.
An example of how you could fill this in: identify a small subset of the problem that is relatively quick and easy to test. If the entirety of the problem can be solved, then for sure this small subset has to be solvable too. If you can't solve this small sub-problem, then you know there's not much point diving into the larger rabbit hole yet. However if you do solve the sub-problem, then that might show you the potential that exists, it may allow you to already look at adjacent problems using the results of this early test, and also important: it will give you additional motivation to keep going.
Modus tollens. If X -> Y, then the only way for Y to be false is if X is false.
Contrapositive (if ~Y, then ~X) is logically equivalent to the original implication (if X, then Y). Instead of proving the latter, you can prove the former contrapositive.
It’s not to test if X works, it’s to test if X has any chance of working.
There are lots of great answers from a problem solving perspective. But anxiety and paralysis are also psychological and shouldn't normally be involved in problem solving.
Anxiety about problems often leads to rumination, which causes more anxiety. It can be a bad feedback loop. In a rumination cycle, people generally don't seek out new information, so their mind spends time analyzing and re-analyzing the same incomplete information. Lots of tricks on this thread really come down to ways to trick yourself into systematically getting more information.
But I think it's also worth taking a step back and looking at the anxiety as a "problem" in itself that can be solved. And as a psychological problem it's amenable to all sorts of treatments like exercise, therapy, etc. So don't forget to take care of your brain!
That said, one of my personal favorite strategies when I find myself ruminating is to put the problem aside and do some curiosity-driven learning that's adjacent to the problem. It both gives me more information and makes my brain feel less antagonism toward the topic. I usually find this is enough to get unstuck.
Anxiety about problems often leads to rumination, which causes more anxiety. It can be a bad feedback loop. In a rumination cycle, people generally don't seek out new information, so their mind spends time analyzing and re-analyzing the same incomplete information. Lots of tricks on this thread really come down to ways to trick yourself into systematically getting more information.
But I think it's also worth taking a step back and looking at the anxiety as a "problem" in itself that can be solved. And as a psychological problem it's amenable to all sorts of treatments like exercise, therapy, etc. So don't forget to take care of your brain!
That said, one of my personal favorite strategies when I find myself ruminating is to put the problem aside and do some curiosity-driven learning that's adjacent to the problem. It both gives me more information and makes my brain feel less antagonism toward the topic. I usually find this is enough to get unstuck.
>> Anxiety about problems often leads to rumination, which causes more anxiety. It can be a bad feedback loop. In a rumination cycle, people generally don't seek out new information, so their mind spends time analyzing and re-analyzing the same incomplete information.
I once got in that cycle, ruminating on possible interpretations of a situation. It can really bring you down. I noticed that each of my possible interpretations produced a different (strong) emotional response and concluded that was driving a "need to know". There is a thing in psychology called "intolerance of uncertainty" and for me the emotional uncertainty was driving the rumination.
So when ruminating, ask yourself "why is this so important to me?" If you can't answer that, get some distance from it before the rumination bring you down. In my case I knew that the answer to "why is this important to me?" was itself very important to understand so I kept at it until I had my answer to that.
I once got in that cycle, ruminating on possible interpretations of a situation. It can really bring you down. I noticed that each of my possible interpretations produced a different (strong) emotional response and concluded that was driving a "need to know". There is a thing in psychology called "intolerance of uncertainty" and for me the emotional uncertainty was driving the rumination.
So when ruminating, ask yourself "why is this so important to me?" If you can't answer that, get some distance from it before the rumination bring you down. In my case I knew that the answer to "why is this important to me?" was itself very important to understand so I kept at it until I had my answer to that.
"analysis paralysis"
https://en.wikipedia.org/wiki/Analysis_paralysis
https://en.wikipedia.org/wiki/Analysis_paralysis
Congratulate your anxiety/paralysis on being right, and do not try to overtly solve the problem. Instead, try moving the situation forward in one or more ways:
- Documentation: Write up the history of the problem, when it occurs, which users are affected, what previous work has been done toward solving the problem, etc.
- Reduction: Only look for limited solutions - proving a few of the "easy" cases for some difficult mathematical theorem. Patches or work-arounds that only reduce the impact of a computer program failing. Chemical synthesis of a less-tricky part of a complex organic molecule.
- Counter-example: A counter-example can be literal - the problem was to prove a mathematical theorem, and your counter-example proves that the theorem is wrong. Or, it can be a proof that the problem is impossibly difficult. The mathematical theorem you were told to prove is equivalent to the Axiom of Choice. The computer programming challenge is equivalent to P vs. NP. The physical theory/device you need to create would violate the laws of thermodynamics.
- Documentation: Write up the history of the problem, when it occurs, which users are affected, what previous work has been done toward solving the problem, etc.
- Reduction: Only look for limited solutions - proving a few of the "easy" cases for some difficult mathematical theorem. Patches or work-arounds that only reduce the impact of a computer program failing. Chemical synthesis of a less-tricky part of a complex organic molecule.
- Counter-example: A counter-example can be literal - the problem was to prove a mathematical theorem, and your counter-example proves that the theorem is wrong. Or, it can be a proof that the problem is impossibly difficult. The mathematical theorem you were told to prove is equivalent to the Axiom of Choice. The computer programming challenge is equivalent to P vs. NP. The physical theory/device you need to create would violate the laws of thermodynamics.
I'd also add:
- Define a very clear target. Include quantities. If the goal is "speed up the api endpoint"... measure the current performance and define a target. The goal must become tangible and measurable in an objective way. Otherwise, it's just up to someone's opinion whether the goal was reached.
- Get stakeholder buy-in on that target. Also communicate your estimate of the probability of success.
- Define a very clear target. Include quantities. If the goal is "speed up the api endpoint"... measure the current performance and define a target. The goal must become tangible and measurable in an objective way. Otherwise, it's just up to someone's opinion whether the goal was reached.
- Get stakeholder buy-in on that target. Also communicate your estimate of the probability of success.
These are really solid suggestions with great examples. The only thing I can think of to add at the moment are a few quotes:
- “The difference between screwing around and science is writing it down.” ― Adam Savage (via Good Reads)
- "Divide and Conquer" ― Many
- “The difference between screwing around and science is writing it down.” ― Adam Savage (via Good Reads)
- "Divide and Conquer" ― Many
I think that Good Reads version is a slight misquote. Adam said "the only difference...".
Great catch!
There is reddit post with imgur link, both from Adam Savage himself that clarifies the correct quote:
"remember kids, the only difference between screwing around and science is writing it down" - ballistics expert Alex Jason
They are both easily findable when searching the correct quote, its a good story. sources: https://www.reddit.com/r/mythbusters/comments/3wgqgv/the_ori... https://imgur.com/1h3K2TT/
There is reddit post with imgur link, both from Adam Savage himself that clarifies the correct quote:
"remember kids, the only difference between screwing around and science is writing it down" - ballistics expert Alex Jason
They are both easily findable when searching the correct quote, its a good story. sources: https://www.reddit.com/r/mythbusters/comments/3wgqgv/the_ori... https://imgur.com/1h3K2TT/
Meanwhile...
Boss: "Hey how are you getting on? You said this'll take two weeks tops, we've got so many clients asking for this thing."
It's most problematic when something looks simple and easy at first glance but then ends up with corner cases upon corner cases and the damn thing refuses to work properly while you sink into a pit of despair amidst calls to deliver already.
Boss: "Hey how are you getting on? You said this'll take two weeks tops, we've got so many clients asking for this thing."
It's most problematic when something looks simple and easy at first glance but then ends up with corner cases upon corner cases and the damn thing refuses to work properly while you sink into a pit of despair amidst calls to deliver already.
Communication is key in this scenario, You've got to be explaining an under estimation as soon as you know it is.
This is more important than you think. To expand on the parent, when communicating keep in mind several extremely important goals:
- you are able to organize your work. If a problem is potentially without a solution, you're likely not going to be able to hold all of that info in your brain at all times.
- when you bring someone in to help, they could get all the context they need by reading the materials - emails, documents, comments, etc, that you already created and organized. They will likely not have all the context you have, but they will have information available and digested.
- it helps keep stakeholders up to date on the progress and risks. Maybe you are able to just talk to your boss, but you don't want to repeat all of that each time you tell someone new. You don't want to setup a meeting just to update your stakeholders
- in case you are able to solve the problem, the materials you write while solving the problem will serve as a staring point for the documentation of the feature/product
- you are able to organize your work. If a problem is potentially without a solution, you're likely not going to be able to hold all of that info in your brain at all times.
- when you bring someone in to help, they could get all the context they need by reading the materials - emails, documents, comments, etc, that you already created and organized. They will likely not have all the context you have, but they will have information available and digested.
- it helps keep stakeholders up to date on the progress and risks. Maybe you are able to just talk to your boss, but you don't want to repeat all of that each time you tell someone new. You don't want to setup a meeting just to update your stakeholders
- in case you are able to solve the problem, the materials you write while solving the problem will serve as a staring point for the documentation of the feature/product
Expectation management. Be predictable in your output even if it means being a bit worse on average. If you occasionally perform miracles, don’t be surprised when people show up asking for more miracles.
I’d start by not telling my boss it’ll take two weeks to solve a problem that might not have a solution.
This is why I prefer to commit to stages of development, rather than a shipping feature, for things that require research. If possible, I will absolutely try to avoid giving a target date for a feature until I've already got a prototype working. I will give a target date for the prototype, though... even though that number is far more often just an arbitrary date than not.
I understand that my approach frustrates managers who demand timelines. But sometimes reality interferes with those timelines, and I'm the one steeped in reality.
I understand that my approach frustrates managers who demand timelines. But sometimes reality interferes with those timelines, and I'm the one steeped in reality.
Deliver an approximate or good-enough solution to buy yourself time.
Which then immediately gets thrown into production and becomes the bedrock upon which further bad decisions are irreversibly built on every time, of course. Now the time pressure is even higher because people are actively complaining about the issues that cannot be solved with the hastily jerry rigged implementation ("should be simple right, just do..." is a bane of my existence).
This comment reminded me of something I read about laymen estimating the difficulty of technical problems. IIRC the two examples were:
a) Given a quality photograph, detect if it's of a bird.
b) For the same photograph, detect if it's taken in a national park.
The first problem may seem much easier if one does not know about EXIF geodata, assuming it's available in this scenario.
a) Given a quality photograph, detect if it's of a bird.
b) For the same photograph, detect if it's taken in a national park.
The first problem may seem much easier if one does not know about EXIF geodata, assuming it's available in this scenario.
This is an inversion of the originating XKCD, which possible shows how far machine learning has come in the intervening years: https://xkcd.com/1425/
No, the first one is still much harder – and still seems easier to the layperson.
Just honesty. "Yes I did, I was wrong and this is why" you might get back "ok! skip that bit, it's not so important now after all"
This might sound like an easy out but I consider that there is a choice of doing nothing. Sunsetting a problem can be as simple as forgetting it exists in the first place. This is not merely "giving up" but rather deciding the best action take at the moment is do actively do nothing about the problem.
Over the past few years when I have had problems that pop up or what really happens is that I go looking for problems, the most often solution has been to literally do nothing.
And having that as an active option when I first start looking at the problem and listing solutions ends up having far more options to me for the problem than if I was like "I MUST SOLVE THIS".
It could also mean "wait" is the best possible action I can take now. And instead of being perturbed by waiting it is an active decision to wait.
Over the past few years when I have had problems that pop up or what really happens is that I go looking for problems, the most often solution has been to literally do nothing.
And having that as an active option when I first start looking at the problem and listing solutions ends up having far more options to me for the problem than if I was like "I MUST SOLVE THIS".
It could also mean "wait" is the best possible action I can take now. And instead of being perturbed by waiting it is an active decision to wait.
I've observed this as well. It's very satisfying to finally realize you don't need to solve it in the first place, because your assumptions created a problem where there was none.
E.g. mulling for weeks over optimizing some code until you realize to measure it as-is and it isn't even slow!
Or maybe there's room in the underlying design to shift the weight off the problem, thus "solving" it laterally (by solving some other, easier, problem instead).
E.g. mulling for weeks over optimizing some code until you realize to measure it as-is and it isn't even slow!
Or maybe there's room in the underlying design to shift the weight off the problem, thus "solving" it laterally (by solving some other, easier, problem instead).
In such cases I've seen (and used) a saying in french that goes:
> Il est urgent d'attendre
which loosely translates to:
> waiting is of utmost urgency
The french quote can be traced to a translation of Asimov's Foundation, but I can't seem to find the original version :/
> Il est urgent d'attendre
which loosely translates to:
> waiting is of utmost urgency
The french quote can be traced to a translation of Asimov's Foundation, but I can't seem to find the original version :/
> The french quote can be traced to a translation of Asimov's Foundation
It's actually much older than that, I read it already in 19th century books; no idea when it first came out.
It's actually much older than that, I read it already in 19th century books; no idea when it first came out.
That's about what I recall as well (18~19th) but could not find a definitive source older than Asimov with my time-limited search
> It could also mean "wait" is the best possible action I can take now. And instead of being perturbed by waiting it is an active decision to wait.
As an example: I worked on a PhD in applying machine learning to certain tasks in programming and mathematics. I ended up burning-out and had to quit.
When I started in 2014, most cutting-edge ML research was on image processing like convolutional neural networks. That's a very bad fit for the sorts of tree-structures and text sequences I wanted to use. The state of the art for the latter were RNNs which are notoriously slow (hard to parallelise), suffer exploding/vanishing gradients (needing e.g. LSTM), etc.
Transformers and LLMs solve the issues I was facing; so in hindsight it would have been better to wait a few years (I believe the Attention Is All You Need paper came out in 2017?)
As an example: I worked on a PhD in applying machine learning to certain tasks in programming and mathematics. I ended up burning-out and had to quit.
When I started in 2014, most cutting-edge ML research was on image processing like convolutional neural networks. That's a very bad fit for the sorts of tree-structures and text sequences I wanted to use. The state of the art for the latter were RNNs which are notoriously slow (hard to parallelise), suffer exploding/vanishing gradients (needing e.g. LSTM), etc.
Transformers and LLMs solve the issues I was facing; so in hindsight it would have been better to wait a few years (I believe the Attention Is All You Need paper came out in 2017?)
Are you Microsoft?
Not sure what kind of problem you are talking about but there are a few things you can try:
Look at it from different angles to understand the problem better. Understand its properties better.
Look for its underlying causes. That is, may be you need to solve different problems!
Nibble at it. That is, solve sub problems so that you are left with a smaller (but possibly harder) problem.
Work on them subconsciously. That is, when you learn something new, let it sink in by walking away and doing something else.
Conversely, focus on the problem 100% when you are working it. If necessary, block out all distractions, including mental ones.
Look at it from a potential solution point of view. That is, if there is a solution, think of what properties it must have.
Never give up! Assuming the problem is worth solving.
Conversely, try to prove it is unsolvable so that you give up on it ASAP!
Keep notes and review them often. You may find clues in something you have already noted.
Try to explain the problem, what you have learned and your attempts to an interested person who can act as your sounding board. The process of explaining will force you to organize your thoughts and the other person can ask questions that will force you to think about cases you may not have considered. Without a sounding board and without writing down your observations you may end up going in circles.
Look at it from different angles to understand the problem better. Understand its properties better.
Look for its underlying causes. That is, may be you need to solve different problems!
Nibble at it. That is, solve sub problems so that you are left with a smaller (but possibly harder) problem.
Work on them subconsciously. That is, when you learn something new, let it sink in by walking away and doing something else.
Conversely, focus on the problem 100% when you are working it. If necessary, block out all distractions, including mental ones.
Look at it from a potential solution point of view. That is, if there is a solution, think of what properties it must have.
Never give up! Assuming the problem is worth solving.
Conversely, try to prove it is unsolvable so that you give up on it ASAP!
Keep notes and review them often. You may find clues in something you have already noted.
Try to explain the problem, what you have learned and your attempts to an interested person who can act as your sounding board. The process of explaining will force you to organize your thoughts and the other person can ask questions that will force you to think about cases you may not have considered. Without a sounding board and without writing down your observations you may end up going in circles.
When the solution is blocked by the diagnosis of the problem, and there is no methodology or algorithm for the diagnosis, I usually resort to a sort of "grid-search", i.e. brute force through the problem space. I once had a weird off-by-one pointer problem in circular buffer in a C++ multi-threaded environment (it was almost 20 years ago), and I had to systematically eliminate code block by code block as possible origin, until I got to a block small enough for me to throw my entire intellect at. I solved it.
I once explained this to an audience using a riddle: "How far can you walk into a forest?" That type of riddle has no method or algorithm for solution. The answer usually "comes to you" or doesn't. But knowing that riddles depend on a play on words or different meanings of words in different contexts, I suggested that one can analyse each word at a time: e.g. "you vs. someone else?", "walk vs. some other way of moving?", "into vs. out of?", "why specifically a forest?" etc. The answer, of course, comes from "into vs. out of" -- you can only walk into a forest till the mid point. After that you're walking out. Not an ideal example, but I always remember it when I'm faced with an intractable problem.
The method also helps stay motivated because there's a sense of progress: you're racking up a count of things that are definitely not the cause of the problem.
I once explained this to an audience using a riddle: "How far can you walk into a forest?" That type of riddle has no method or algorithm for solution. The answer usually "comes to you" or doesn't. But knowing that riddles depend on a play on words or different meanings of words in different contexts, I suggested that one can analyse each word at a time: e.g. "you vs. someone else?", "walk vs. some other way of moving?", "into vs. out of?", "why specifically a forest?" etc. The answer, of course, comes from "into vs. out of" -- you can only walk into a forest till the mid point. After that you're walking out. Not an ideal example, but I always remember it when I'm faced with an intractable problem.
The method also helps stay motivated because there's a sense of progress: you're racking up a count of things that are definitely not the cause of the problem.
Great explanation of divide and conquer for any situation. Thanks!
I know exactly what you mean. I worked for years in research in a problem I was not sure it could be solved. Here is the advice I could offer you:
- Determine what is the problem. Easier said than done. You most likely don't understand the problem. Finding the right abstractions to understand what is the problem is half the work. Focus on that a lot early on.
- You will not solve the problem by sitting for two hours and trying to think of a solution. Accept that. If it is a hard problem, it will take you months of thinking, writing prototypes/solutions, trying different angles. And then, at a random moment it will click, and the solution will feel obvious.
- Iterate a lot. Start with something small, solve it, and do it again and again. Accept that you will fail hundreds of times until finding the right solution. Try to make the process enjoyable. If it is a research project, break it into parts, where the solution of each small part provides value (a paper), so that you can enjoy some success that keep you working. If it is a startup, build products that provide value and are in the road to solving the big product.
- Some problems can be solved in a phd, some in a career, some in a generation. If you are targeting poverty, accept that you will spend your life on that with the hope of making small progress.
- Determine what is the problem. Easier said than done. You most likely don't understand the problem. Finding the right abstractions to understand what is the problem is half the work. Focus on that a lot early on.
- You will not solve the problem by sitting for two hours and trying to think of a solution. Accept that. If it is a hard problem, it will take you months of thinking, writing prototypes/solutions, trying different angles. And then, at a random moment it will click, and the solution will feel obvious.
- Iterate a lot. Start with something small, solve it, and do it again and again. Accept that you will fail hundreds of times until finding the right solution. Try to make the process enjoyable. If it is a research project, break it into parts, where the solution of each small part provides value (a paper), so that you can enjoy some success that keep you working. If it is a startup, build products that provide value and are in the road to solving the big product.
- Some problems can be solved in a phd, some in a career, some in a generation. If you are targeting poverty, accept that you will spend your life on that with the hope of making small progress.
> Iterate a lot. Start with something small, solve it, and do it again and again.
> Accept that you will fail hundreds of times until finding the right solution.
I guess it's easier with problems that don't destroy their own launch pad if something goes wrong.
Oversimplify until you can solve a toy version and then try to see if it has an extension to the full problem or if there is some reason it can’t be extended. Sometimes you’ll learn the core reason it’s unsolvable, other times you’ll solve it. Other other times, you’ll be right back where you started but with just a little more wisdom.
The human brain is usually quite adapted to iterating on some existing thing vs summoning a novel solution from the void of unexistance. So, just find Some starting point.
Polya even talks about this in his famous book “How To Solve It”: “If you cannot solve the proposed problem”
The human brain is usually quite adapted to iterating on some existing thing vs summoning a novel solution from the void of unexistance. So, just find Some starting point.
Polya even talks about this in his famous book “How To Solve It”: “If you cannot solve the proposed problem”
Maybe you are aiming at a different audience, but this conundrum is similar to how most research problems in pure sciences go -- problem might seem interesting, but finding the solution is hard and sometimes we don't even know if a solution exists. Mathematicians often use the words "existence" and "uniqueness": Does a solution to the problem exist and if it does, is it unique? George Polya's How to Solve It consists of essays on how to approach problems in general (not necessarily exclusive to mathematics in my opinion). You can find an excellent summary at https://math.berkeley.edu/~gmelvin/polya.pdf.
Essentially, the advice boils down to:
1. use the defining features of the set up
2. give things/variables meaningful names
3. leverage symmetry
4. try describing one object in two different ways
5. draw a picture, flowchart to visually represent
6. ask a simpler version of the problem by removing some constraints or considering very specific cases
7. read a lot and think about problems a lot
Essentially, the advice boils down to:
1. use the defining features of the set up
2. give things/variables meaningful names
3. leverage symmetry
4. try describing one object in two different ways
5. draw a picture, flowchart to visually represent
6. ask a simpler version of the problem by removing some constraints or considering very specific cases
7. read a lot and think about problems a lot
Nowadays, I utilize GPT-4's API for nearly every problem I encounter. By inputting all the relevant information and applying different prompts, I gain a clearer understanding to make informed decisions. Despite being released less than a year ago, I'm astounded (to say the least) at how integral GPT has become to my thinking process.
Prompts that I use that significantly aid the process:
Prompts that I use that significantly aid the process:
* Provide a concise definition of [specific topic or concept] and explain its key characteristics.
* List three advantages and three disadvantages of [specific technology, method, or approach].
* Explain the step-by-step process of [specific task or procedure] in a clear and logical manner.
* Compare and contrast [two different approaches, methods, or models] in terms of their strengths and limitations.
* Predict the potential impact of [emerging technology or trend] on [specific industry or domain].
* Describe the main challenges associated with [problem or issue] and propose possible solutions.
* Summarize the main findings and conclusions of [research paper or study] in three concise points.
* Create a comprehensive list of resources, including books, articles, and websites, related to [specific topic].
* Provide examples of real-world applications or use cases for [specific technology or methodology].
* Offer insights and recommendations for optimizing [specific process or system] based on industry best practices.For cases where you get "new" information from it, what's your verification process to guard against hallucinations?
Imagine that GPT is just another person you communicate with. When they give you new information, how do you guard against them being possibly wrong? You verify the information by other sources.
The "quality" of the wrong information you get from GPT-4 is very different from a human who is wrong. For example, I wouldn't expect a human to give me a long list of books that don't actually exist without hesitation.
Sure, but still, if you ask GPT/person "What are the best books about teaching dogs to sit?" you'd still look up each book individually, read reviews and figure out if they really are worth the time reading, before starting to purchasing the books. And you'd find out if the book exists or not as soon as you search.
So even if the "quality" is different, the way to verify the information is the same.
So even if the "quality" is different, the way to verify the information is the same.
Both AI and humans can be wrong, but in different ways. Humans often mess up due to bias or memory slips, while AI usually stumbles due to data gaps or misunderstood context. AI misinformation isn't 'worse,' it's just different. Understanding this helps us use AI more effectively.
[deleted]
Zero trust, you have to unit test, run what it gives you, tell it in a separate session a co worker gave you this solution, it doesn't work but explain why. I quite often enlist the bot in helping to prove itself right.
Googling. Just today I asked Chat-GPT (not 4) for papers or books about some topic, and it gave me five pointers. Two of them contained useful information, the rest was hallucinated.
Use the GPT4 with web browsing mode enabled or Bing Chat if you want links to real articles. Bing Chat has come a long, long ways. Impressive capabilities. Much less hallucination.
Bing chat? You mean having to use Edge, aka Chromium without any extensions? I'd sooner go to Firefox.
GPT 4 with browsing isn't quite there yet either, usually takes at least two or three attempts to not have it fail somewhere in between. Should be pretty good once they iron it out though.
GPT 4 with browsing isn't quite there yet either, usually takes at least two or three attempts to not have it fail somewhere in between. Should be pretty good once they iron it out though.
https://github.com/dice2o/BingGPT
You don't need to use Edge to be able to utilize Bing Chat :)
https://github.com/sunner/ChatALL
This one let's you send to 9 models simultaneously and in parallel. Interesting the differing responses for each.
You don't need to use Edge to be able to utilize Bing Chat :)
https://github.com/sunner/ChatALL
This one let's you send to 9 models simultaneously and in parallel. Interesting the differing responses for each.
I usually tell it to add confidence score of whether it's real and to prioritize real stuff
Play. We often use that word to mean "not serious" or "not work", but children's play is an effort to do exactly what you are talking about - to understand what is possible. Humans are among the most adaptive creatures known. We adapt by trying things, not being successful and then playing with possibilities. If you have anxiety/paralysis about that process then I suspect that is not self-inflicted but inflicted by other people or the institutions you participate in.
There is another factor that causes problems for many people. They don't really give a <insert-obscentity-here> for what they are doing. Bad jobs, stupid ways of doing things, senseless tasks. Just say no. :-)
And finally, we often forget how physically competent we are. Humans can run and walk for hours and hours once they get use to it. Go walk 10 miles. Or better yet run. It helps.
There is another factor that causes problems for many people. They don't really give a <insert-obscentity-here> for what they are doing. Bad jobs, stupid ways of doing things, senseless tasks. Just say no. :-)
And finally, we often forget how physically competent we are. Humans can run and walk for hours and hours once they get use to it. Go walk 10 miles. Or better yet run. It helps.
The original post does not specify the nature of the problem. The problem might be something objective like scientific research. Or something deeply personal like an illness of a friend or family. Depending on the problem, the approach will of course be completely different.
In case of scientific research also, it depends on the situation. A PhD student's approach to a difficult problem with apparantly no solution would be and should be different from someone who does not have a deliverable. If you are a student, you need to deliver some result for a degree. So after some time it would be better to forget about a seemingly unsolvable problem and focus on a new problem which you can solve in perhaps a better way.
A senior researcher however in my opinion should persistently try to solve the harder problems. As I feel it is their job.
Coming back to different problems than scientific research, I can only think of one approach. Ask the question, how important is this problem to you and your life? If it is as important as the the well being of you, or your close ones, then you have no choice but to focus and fight to find a solution.
In case of scientific research also, it depends on the situation. A PhD student's approach to a difficult problem with apparantly no solution would be and should be different from someone who does not have a deliverable. If you are a student, you need to deliver some result for a degree. So after some time it would be better to forget about a seemingly unsolvable problem and focus on a new problem which you can solve in perhaps a better way.
A senior researcher however in my opinion should persistently try to solve the harder problems. As I feel it is their job.
Coming back to different problems than scientific research, I can only think of one approach. Ask the question, how important is this problem to you and your life? If it is as important as the the well being of you, or your close ones, then you have no choice but to focus and fight to find a solution.
Flip it around: the only really interesting problems to work on are the ones where you don't know if there is a solution.
Minimise work on problems with an obvious solution and identify working on the ones that are more mysterious as the most valuable work there is to do. Make a habit of extracting the most from the process, even if it didn't end up in a solution. For example: writing down (and sharing with others) what was learned.
Psychologically, you need enough repeated positive reinforcement, where you work on a problem, end up not solving it, extract the most learning, get recognition from yourself and from others that it was worth the effort. After enough itterations it starts feeling better.
Minimise work on problems with an obvious solution and identify working on the ones that are more mysterious as the most valuable work there is to do. Make a habit of extracting the most from the process, even if it didn't end up in a solution. For example: writing down (and sharing with others) what was learned.
Psychologically, you need enough repeated positive reinforcement, where you work on a problem, end up not solving it, extract the most learning, get recognition from yourself and from others that it was worth the effort. After enough itterations it starts feeling better.
I feel an obvious solution many are missing is simply to ask your colleagues for help. They can either listen to your rambling or they can offer answers when ones you've thought of before or novel strange ones you had dismissed but bear rethinking. If your paralysis is due to you working alone, then your open mindedness should make your amenable to asking for help.
I can attest to this. Fresh inputs are often the only way to get out of a mental feedback loop.
Assuming the context is for solving real-world problems, not textbook problems:
1. Try to solve the smallest (but similar) problem you can think of. You'll learn a lot along the way and might figure out if it is doable or not. As a reminder, just because a problem is solvable doesn't mean it's solvable at an acceptable level of cost/time/effort. Solving a similar, but smaller problem will help you estimate the feasibility of the larger problem.
2. If the problem is worth solving, it's probably affecting people who in turn have tried to solve it before and are using either approximations or imperfect solutions. Talk to them, understand what they've tried, what works and what doesn't. If the problem affects enough people It's very likely that a solution exists already, it just hasn't been generalised, productised, or automated yet.
1. Try to solve the smallest (but similar) problem you can think of. You'll learn a lot along the way and might figure out if it is doable or not. As a reminder, just because a problem is solvable doesn't mean it's solvable at an acceptable level of cost/time/effort. Solving a similar, but smaller problem will help you estimate the feasibility of the larger problem.
2. If the problem is worth solving, it's probably affecting people who in turn have tried to solve it before and are using either approximations or imperfect solutions. Talk to them, understand what they've tried, what works and what doesn't. If the problem affects enough people It's very likely that a solution exists already, it just hasn't been generalised, productised, or automated yet.
Here's my general process for problem solving when operating with incomplete information.
Evaluate the severity of the problem first. This usually falls into a bell-curve with the X axis starting at "not a problem" to "armageddon." Attempt to gauge the timeliness of the problem. This tends to fall between "already happened" to "years in the future."
Comparing the severity with the timeliness will give you a good idea of the urgency of the situation, which is then compared to current priorities.
Gather the information you can given the urgency constraints that can give you the best understanding of the problem and accept that this information may necessarily be incomplete.
Recognize that many times you don't have to solve a high-urgency problem but can instead pursue a partial course of action that increases the timeliness, which reduces the urgency as well as provides you more time to gather information to reduce the unknowns.
When deciding a course of action with incomplete information you need to commit to that decision strongly, but always be looking for evidence that the decision is incorrect. Until you have such evidence, stay with your decision. If you find such evidence, re-evaluate since you have the ability to make a more informed decision now. This is generally referred to as "strong opinions, weakly held."
One thing I very much try to do when executing a plan that is based on incomplete information is to have an exit strategy. Try to make choices that give you the ability to roll back changes and restore to the previous state if things go wrong. This will give you more confidence in committing to action.
Always remember that discovering a choice you've made is incorrect is valuable because it increases your knowledge and often leads to greater understanding in an environment in which you lack sufficient information. Don't fear learning this way, it's too valuable to avoid.
Evaluate the severity of the problem first. This usually falls into a bell-curve with the X axis starting at "not a problem" to "armageddon." Attempt to gauge the timeliness of the problem. This tends to fall between "already happened" to "years in the future."
Comparing the severity with the timeliness will give you a good idea of the urgency of the situation, which is then compared to current priorities.
Gather the information you can given the urgency constraints that can give you the best understanding of the problem and accept that this information may necessarily be incomplete.
Recognize that many times you don't have to solve a high-urgency problem but can instead pursue a partial course of action that increases the timeliness, which reduces the urgency as well as provides you more time to gather information to reduce the unknowns.
When deciding a course of action with incomplete information you need to commit to that decision strongly, but always be looking for evidence that the decision is incorrect. Until you have such evidence, stay with your decision. If you find such evidence, re-evaluate since you have the ability to make a more informed decision now. This is generally referred to as "strong opinions, weakly held."
One thing I very much try to do when executing a plan that is based on incomplete information is to have an exit strategy. Try to make choices that give you the ability to roll back changes and restore to the previous state if things go wrong. This will give you more confidence in committing to action.
Always remember that discovering a choice you've made is incorrect is valuable because it increases your knowledge and often leads to greater understanding in an environment in which you lack sufficient information. Don't fear learning this way, it's too valuable to avoid.
Some of the world's greatest inventions were created only after much trial and error by a very persistent person. They were often told that what they were trying to do was impossible and to just give it up. Only after they succeeded were they recognized as a genius instead of as a fool.
Today's mantra of 'fail fast' discourages the kind of steady work required to solve some problems. If a solution can't be found in a very short time, they tell you to give up. This can be beneficial in some cases, but some problems are just not easily solved.
How you deal with this, really is dependent on what kind of person you are. If you are patient and willing to stick with something over a long time, then you could create something truly unique. If you get too anxious when a quick solution eludes you, maybe go with the fail fast method.
Today's mantra of 'fail fast' discourages the kind of steady work required to solve some problems. If a solution can't be found in a very short time, they tell you to give up. This can be beneficial in some cases, but some problems are just not easily solved.
How you deal with this, really is dependent on what kind of person you are. If you are patient and willing to stick with something over a long time, then you could create something truly unique. If you get too anxious when a quick solution eludes you, maybe go with the fail fast method.
Worth pointing out that sometimes things really are impossible. If an MBA tells you something is impossible, it is probably just really hard. If a physicist tells you your idea violates conservation of energy, it is probably not worth wasting your life chasing it.
Survivorship bias at play here. World's greatest inventors who happened to work relentlessly at something important. Although I am sure they had their own good reasons to believe in it.
1. I try to break the problem into smaller problems.
2. Pick the hardest one from those smaller problems and apply step 2 until it can't be broken into smaller pieces.
3. Once I have a "constrained" hard problem, I just try to solve it however I can.
If I solve that problem, that normally allows me to build from there, otherwise I may have to re-frame the problem. But by just starting, I get more information and ideas on how to tackle it.
If I solve that problem, that normally allows me to build from there, otherwise I may have to re-frame the problem. But by just starting, I get more information and ideas on how to tackle it.
This is my approach as well, and I think it's generally considered a best practice in engineering, at least on the software side.
When faced with what sounds like an insurmountable,"where the heck do I even begin?" type of challenge, I like to slice it up and get the obvious bits working, just to build confidence & domain knowledge, while my brain autonomously digests the main problem.
When faced with what sounds like an insurmountable,"where the heck do I even begin?" type of challenge, I like to slice it up and get the obvious bits working, just to build confidence & domain knowledge, while my brain autonomously digests the main problem.
Your question is my argument against puzzle questions in interviews.
When you see a seemingly impossible question in an interview. You know there’s a trick. You’ve been handed a riddle to be solved.
When my boss hands me a seemingly impossible problem, it might actually be impossible. At best it’s a mystery, at worst an enigma, or a paradox. It’s a very different skill set to go searching for a solution that is not obvious and not found in the literature (because nobody wrote it down or they used jargon nobody else knows).
I have a coworker who says things like, “we can do anything in software” if he says it again I’m getting him a biography of Claude Shannon, because apparently he did not learn about Information Theory in school.
When you see a seemingly impossible question in an interview. You know there’s a trick. You’ve been handed a riddle to be solved.
When my boss hands me a seemingly impossible problem, it might actually be impossible. At best it’s a mystery, at worst an enigma, or a paradox. It’s a very different skill set to go searching for a solution that is not obvious and not found in the literature (because nobody wrote it down or they used jargon nobody else knows).
I have a coworker who says things like, “we can do anything in software” if he says it again I’m getting him a biography of Claude Shannon, because apparently he did not learn about Information Theory in school.
It's easy: Expect failure. That doesn't mean you stop. It means you stop worrying about success and just play around with the problem. Poke at it from as many different directions as you can, and keep note of what does not work. Eventually you'll either stumble onto a solution or prove that it is in fact impossible, and your experiments were a success.
"Expect failure" sounds very motto-y. In Latin, so you can use it for your coat of arms, would be: expecto defectum. I suppose one could even operationalize it further: deploy failure, displicare defectum.
This. Keep probing until you’re able to form hypotheses, and keep testing those. Worst case you learn; best case, with each experiment you change something for the better.
One thing I would add is the perspectives and ideally the participation of other people. Absolutely essential if the challenge has any kind of social dimension.
One thing I would add is the perspectives and ideally the participation of other people. Absolutely essential if the challenge has any kind of social dimension.
(assuming math-ish problems) Try to solve a simpler version of the problem. Keep making simplifying assumptions until you get something you can handle.
Often you realize that some parts of the original problem statement were adding complexity with no real benefit. And the techniques you use for the simple version are likely still relevant for the big version.
If the simplified version is still hopelessly difficult, it might be a good starting point for a counterexample/lower bound to convince yourself that the full problem is hard.
Often you realize that some parts of the original problem statement were adding complexity with no real benefit. And the techniques you use for the simple version are likely still relevant for the big version.
If the simplified version is still hopelessly difficult, it might be a good starting point for a counterexample/lower bound to convince yourself that the full problem is hard.
The inverse of this strategy also works well sometimes: try to solve a more general (i.e. "harder") version of the problem. Examples:
* https://math.stackexchange.com/questions/899109/problems-tha...
* https://mathoverflow.net/questions/21214/particular-problem-...
The second link has the neat particular example of showing that something is nonzero by showing that it is odd, which I quite like.
* https://math.stackexchange.com/questions/899109/problems-tha...
* https://mathoverflow.net/questions/21214/particular-problem-...
The second link has the neat particular example of showing that something is nonzero by showing that it is odd, which I quite like.
Agreed!
"This problem is unsolvable" or "Approaches X, Y and Z are not solutions to this problem" are perfectly valid and useful contributions to science and society in general.
Or in the words of Thomas Edison: “I have not failed. I've just found 10,000 ways that won't work.”
The attainability of a solution is the product of a well-defined problem. So focus less on the "have I solved this" and focus more on the "have I made useful progress in defining the problem".
Or in the words of Thomas Edison: “I have not failed. I've just found 10,000 ways that won't work.”
The attainability of a solution is the product of a well-defined problem. So focus less on the "have I solved this" and focus more on the "have I made useful progress in defining the problem".
If you're talking about dealing with the stress of a problem unanswered, my counter-intuitive go-to strategy is to think of the worst possible outcomes of me not coming up with a solution. When I think of clear consequences and realize I can deal with them (even if they are unpleasant) and it takes the edge off so I can think more clearly. The consequences are rarely world-ending, after all.
As for strategies for finding solutions for complex/potentially unsolvable problems:
- Try to sketch it out. When thinking about complex problems, we tend to spiral and sometimes they seem too abstract when contained in our heads, putting them to paper makes them more comprehensible and less overwhelming. Even if you can't sketch out the whole problem, better understanding parts of it makes it easier to understand the whole
- Explain it to someone else. Aside from the benefit of the rubber duck approach, talking it out with people with different backgrounds and cognitive processes helps you look at it differently. Maybe you're a deep narrow thinker and someone who is a shallow wide thinker will help you se the whole picture or vice versa
- Can you break it up into smaller components? If yes, great! Even if you don't solve all the components, you'll still end up with a smaller problem then you started with. And solving some parts of it may change it all together.
- Review the context and everything surrounding the problem. My colleague recently mentioned that when they get a ticket for an issue, they often end up fixing or upgrading a completely different part of the system than the one where issue occurs, and that issue just falls into place
As for strategies for finding solutions for complex/potentially unsolvable problems:
- Try to sketch it out. When thinking about complex problems, we tend to spiral and sometimes they seem too abstract when contained in our heads, putting them to paper makes them more comprehensible and less overwhelming. Even if you can't sketch out the whole problem, better understanding parts of it makes it easier to understand the whole
- Explain it to someone else. Aside from the benefit of the rubber duck approach, talking it out with people with different backgrounds and cognitive processes helps you look at it differently. Maybe you're a deep narrow thinker and someone who is a shallow wide thinker will help you se the whole picture or vice versa
- Can you break it up into smaller components? If yes, great! Even if you don't solve all the components, you'll still end up with a smaller problem then you started with. And solving some parts of it may change it all together.
- Review the context and everything surrounding the problem. My colleague recently mentioned that when they get a ticket for an issue, they often end up fixing or upgrading a completely different part of the system than the one where issue occurs, and that issue just falls into place
Paraphrasing Dune, fear is the mind-killer. It doesn't help to search for alternate points of views or approaches as you are fixed in one possible outcome.
Learning more about the problem and the system where it happens may give a different perspective, a broader point of view, another attack angle. Or, in the worst case, realize that is not that critical, that it can be redefined, or adapt around it.
Learning more about the problem and the system where it happens may give a different perspective, a broader point of view, another attack angle. Or, in the worst case, realize that is not that critical, that it can be redefined, or adapt around it.
Oh yeah, these are basically my job. I have a few ways:
1. Set a time budget and be willing to walk away if I'm not making progress. Sometimes "yep, sucks, but is what it is" is the answer.
2. If it looks impossible, be more willing than usual to reframe it. If it's a detail problem, zoom out and see if the bigger picture offers you a way around. If it's a big picture problem, challenge the assumptions. If it's still impossible, see 1.
3. Involve other people with other points of view early in the process. There are a bunch of different teams that work in sync to get anything out the door at my company. Getting any difficult problem solved will usually involve understanding the work of at least two of them. The challenge with potentially unsolveable problems is knowing which two.
4. I cannot reiterate number 1 enough. Sometimes a problem might have a solution, but I just have to be ok with the fact that I'm not going to be the one to find it. Maybe the next problem I tackle will present me with a nicely reframed version of this problem for me to go back and take on.
1. Set a time budget and be willing to walk away if I'm not making progress. Sometimes "yep, sucks, but is what it is" is the answer.
2. If it looks impossible, be more willing than usual to reframe it. If it's a detail problem, zoom out and see if the bigger picture offers you a way around. If it's a big picture problem, challenge the assumptions. If it's still impossible, see 1.
3. Involve other people with other points of view early in the process. There are a bunch of different teams that work in sync to get anything out the door at my company. Getting any difficult problem solved will usually involve understanding the work of at least two of them. The challenge with potentially unsolveable problems is knowing which two.
4. I cannot reiterate number 1 enough. Sometimes a problem might have a solution, but I just have to be ok with the fact that I'm not going to be the one to find it. Maybe the next problem I tackle will present me with a nicely reframed version of this problem for me to go back and take on.
Are you talking about a technical problem, or a conceptual problem?
By technical, I mean something like "The service keeps crashing roughly once a week and we can't figure it out."
By conceptual, something like "How do I hire the best engineers within my tiny budget?" or "How do we get SOC2 certification even though we do this or that poorly?"
In either case, I always start with building an understanding of the problem. Gather what you do know about it. Write down why it is hard. Note any contradictions. Note things you don't know, such as terminology, concepts, or data that is not available.
Next, research the things you don't know. Look up the terminology. Read up on the concepts. See what you need to do to get the data. At this point, you don't need perfect answers. Anything that improves your knowledge and understanding is a step in the right direction.
At this point you should be able to talk about the problem in clear terms. It should be clear what you know and what you don't know. Tackle the unknowns now. Take steps to make them known. Read. Talk to people. Observe.
At some point you might run into contradictions. Think long and hard if they are really contradictions, or if you are making some assumptions incorrectly which are leading to a contradiction. What if you change some assumptions? Does the contradiction go away?
If the contradictions seem firm, see if you can redefine the problem ina different way that removes the contradiction. Solving this other problem might not solve the original problem, but it might make the situation better. It might mitigate the problem.
This is not a straight path. It's applying the learning loop iteratively. Assess where you are. Decide on a next step to change something. Take those steps. Repeat.
By technical, I mean something like "The service keeps crashing roughly once a week and we can't figure it out."
By conceptual, something like "How do I hire the best engineers within my tiny budget?" or "How do we get SOC2 certification even though we do this or that poorly?"
In either case, I always start with building an understanding of the problem. Gather what you do know about it. Write down why it is hard. Note any contradictions. Note things you don't know, such as terminology, concepts, or data that is not available.
Next, research the things you don't know. Look up the terminology. Read up on the concepts. See what you need to do to get the data. At this point, you don't need perfect answers. Anything that improves your knowledge and understanding is a step in the right direction.
At this point you should be able to talk about the problem in clear terms. It should be clear what you know and what you don't know. Tackle the unknowns now. Take steps to make them known. Read. Talk to people. Observe.
At some point you might run into contradictions. Think long and hard if they are really contradictions, or if you are making some assumptions incorrectly which are leading to a contradiction. What if you change some assumptions? Does the contradiction go away?
If the contradictions seem firm, see if you can redefine the problem ina different way that removes the contradiction. Solving this other problem might not solve the original problem, but it might make the situation better. It might mitigate the problem.
This is not a straight path. It's applying the learning loop iteratively. Assess where you are. Decide on a next step to change something. Take those steps. Repeat.
It could also be a "wicked problem" where there is no "right or wrong" answer, each attempt at solving the problem affects how the next attempt could be done and usually you can't even understand the problem before starting on a solution. Or where each attempt is so expensive that you can't afford a failure. When I ran for elected office, one of the problems was the light rail expansion project. If the route "doesn't work", you can't rip up the rails and try again. Not at > $2 billion per route. Highways are similar, as one gets built, it alters the community so much that you can't do any sort of "trial and error".
In the case of wicked problems, lots of meetings with all the shareholders followed by lots of simulations.
0 - https://en.wikipedia.org/wiki/Wicked_problem
In the case of wicked problems, lots of meetings with all the shareholders followed by lots of simulations.
0 - https://en.wikipedia.org/wiki/Wicked_problem
I know exactly what you're talking about I think. My problems are solved by code such that code will have something to do with the approach. With that in mind I tend to write lots of little one off programs to test a particular idea or see if a particular phenomenon exists in a set for example. In building these I'll version them in the filename ( XXX_Test_V1.sh ) such that I can compare runs and later maybe chain some runs together.
The point here is just to get moving in some direction, get familiar with your data and start testing some assumptions. I learned a lot about my problem from doing this and eventually came up with some testable theory.
Forget architecture, "elegant code", time complexity, "does it scale" etc - just write things that give you answers or allow you to gain some insight and do so with the minimum of effort spent on design. If you ever find yourself onto something you can refactor then. For me the main thing is to get the learning process started.
The point here is just to get moving in some direction, get familiar with your data and start testing some assumptions. I learned a lot about my problem from doing this and eventually came up with some testable theory.
Forget architecture, "elegant code", time complexity, "does it scale" etc - just write things that give you answers or allow you to gain some insight and do so with the minimum of effort spent on design. If you ever find yourself onto something you can refactor then. For me the main thing is to get the learning process started.
I have taken on intractable problems and succeeded in finding novel solutions to them (with a team) Here's some advice.
Don't give up before at least you:
1. Prototype many solutions, and identify their "dead ends" through engagement with reality.
2. Share your progress as widely as you can - If at a company, share your progress internally once you reach a dead end / challenge you can't overcome. "Many minds make light work". Your challenge might not be the same as a colleague or leader has encountered before, but it may be similar in some way, and there may be a smaller piece of the puzzle that they have solved before. Have faith that creative solutions you could never have anticipated at first most likely will emerge, but maybe not first from your own mind. If a public project, share as widely as you can. When sharing, skip to the dead-end part quickly so you can spark the minds of your audience and engage them in the search for solutions.
3. Use true "design thinking" which is abductive reasoning and reframing. For more on what exactly that is, and how to do it, I recommend the writing of Kees Dorst in his book "Frame Innovation". "Universal Traveler" is another good resource.
https://www.amazon.com/Frame-Innovation-Create-Thinking-Desi...
https://www.amazon.com/Universal-Traveler-Soft-Systems-Creat...
Here's a good video of a talk by Kees Dorst that can help you understand what Reframing is: https://www.youtube.com/watch?v=j60jMV00kfk
Don't give up before at least you:
1. Prototype many solutions, and identify their "dead ends" through engagement with reality.
2. Share your progress as widely as you can - If at a company, share your progress internally once you reach a dead end / challenge you can't overcome. "Many minds make light work". Your challenge might not be the same as a colleague or leader has encountered before, but it may be similar in some way, and there may be a smaller piece of the puzzle that they have solved before. Have faith that creative solutions you could never have anticipated at first most likely will emerge, but maybe not first from your own mind. If a public project, share as widely as you can. When sharing, skip to the dead-end part quickly so you can spark the minds of your audience and engage them in the search for solutions.
3. Use true "design thinking" which is abductive reasoning and reframing. For more on what exactly that is, and how to do it, I recommend the writing of Kees Dorst in his book "Frame Innovation". "Universal Traveler" is another good resource.
https://www.amazon.com/Frame-Innovation-Create-Thinking-Desi...
https://www.amazon.com/Universal-Traveler-Soft-Systems-Creat...
Here's a good video of a talk by Kees Dorst that can help you understand what Reframing is: https://www.youtube.com/watch?v=j60jMV00kfk
I had this problem near the end of my PhD (physics).
Because of a sad conjecture of circumstances I ended up having a problem to resolve quickly. The more I was thinking about it, the harder it looked.
I then went to some people who were also physicists (not necessarily in my specialty) to discuss it, and see how it goes. It was confirmed that the problem was indeed hard.
So I did the only think which made sense in that context: I simplified it. And was crossing fingers to still have a more or less working model, with known limitations.
Which ended up not being that easy, but was ok-ish - and solvable.
I then want for dinner with a friend, and by main course she casually suggested a completely different approach. It flipped my depressed world around.
So to summarize the advices: see if the problem is really hard, then discuss with clever people to see if they feel the same, then simplify. Or take a few steps back and consider looking at it though a completely different perspective.
Good luck!
Because of a sad conjecture of circumstances I ended up having a problem to resolve quickly. The more I was thinking about it, the harder it looked.
I then went to some people who were also physicists (not necessarily in my specialty) to discuss it, and see how it goes. It was confirmed that the problem was indeed hard.
So I did the only think which made sense in that context: I simplified it. And was crossing fingers to still have a more or less working model, with known limitations.
Which ended up not being that easy, but was ok-ish - and solvable.
I then want for dinner with a friend, and by main course she casually suggested a completely different approach. It flipped my depressed world around.
So to summarize the advices: see if the problem is really hard, then discuss with clever people to see if they feel the same, then simplify. Or take a few steps back and consider looking at it though a completely different perspective.
Good luck!
The title and the first question are two different things.
If the "anxiety/paralysis" is the real issue here, then it is a completely different topic that probably needs external help.
For a "problem you are not sure has a solution", if it's related to job/technology, I search for a workaround (if applicable which, most of the times, it is).
If the "anxiety/paralysis" is the real issue here, then it is a completely different topic that probably needs external help.
For a "problem you are not sure has a solution", if it's related to job/technology, I search for a workaround (if applicable which, most of the times, it is).
Recently I just started asking ChatGPT.
It does a very good job guiding me in areas where I'm less familiar.
That's assuming we're talking about technical problems. For non-technical problems I'd reach out to friends (or even better - a therapist). But asking on HN and/or Reddit might be a good idea too, depending on the context.
It does a very good job guiding me in areas where I'm less familiar.
That's assuming we're talking about technical problems. For non-technical problems I'd reach out to friends (or even better - a therapist). But asking on HN and/or Reddit might be a good idea too, depending on the context.
I stare at the problem for hours until it's suddenly dark outside, then if necessary I repeat the following day and the next until I see a solution. I mean this in a quite literal way, and it's in no way not meant as a joke answer.
Surprisingly often this method works.
Surprisingly often this method works.
I would think about different aspects of the problem a lot. How they manifest themselves, which of them I could possibly tackle and which are beyond my comprehension at the moment. Eventually I get enough resolve to approach and make an attempt on those that seem reachable. This almost inevitably clears up just a bit of the nature of other aspects, so that after another period of contemplation you can in turn tackle on them.
So basically, look hard at the problem, chisel it a bit and back off for a moment. If you can't solve it right away it's a substantial task that has to be treated with respect. Naturally this process is poorly suited for meeting deadlines.
So basically, look hard at the problem, chisel it a bit and back off for a moment. If you can't solve it right away it's a substantial task that has to be treated with respect. Naturally this process is poorly suited for meeting deadlines.
Will probably get down voted or ridiculed. There are many engineers/scientists here how will give the logic ways (divide and conquer, research etc).
But here are some non logical ways: - reframing - ignoring/delaying - accepting - praying - delegating
Or what the stoic/religion used to say
God, grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference.
P.s I ain't religious or superstition but there is some wisdom in those teachings.
Now in the context of a project if you really can't find any solution, then document and communicate your findings until the budget runs out..
But here are some non logical ways: - reframing - ignoring/delaying - accepting - praying - delegating
Or what the stoic/religion used to say
God, grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference.
P.s I ain't religious or superstition but there is some wisdom in those teachings.
Now in the context of a project if you really can't find any solution, then document and communicate your findings until the budget runs out..
I am not very religious but sometimes I pray if I have some "unsolvable" problem... No idea if it helps but it feels good to know that maybe "someone" will help me. I do this usually after day long debugging or problem solving without any result, right before bed.
It would help quite a bit if you were able to give a good example.
Lots of good tactics here, but ultimately I think everything boils down to understand the problem better.
Your solution will only come from superior understanding of the problem, if there is one. It probably won't come from testing different solutions.
Almost all software starts with "what is the intput and what is the output" and then understanding how the input leads to the output. If you don't know the input and the output to your system, then you don't have understanding.
Even in your text, you have a "solution" based frame to your statement. Subordinating the problem to the solution rather than the solution to the problem puts so much focus on the solution that the problem itself gets defined in terms of the solution. I have seen feature developers play this out time and time and time again. When the problem gets redefined in terms of the solution, often the original problem remains resulting in increased complexity at little additional value.
So I would start by understanding that not being sure a problem has a solution means you don't understand the problem and its context enough to even be asking if it has a solution which means you should be asking "how do you better understand problems?" or "when do you stop investing in understanding problems?" or "what is a good time trade off between 'understanding problems' and 'directly handling business concerns'"?
When you understand the problem, you'll probably come to the realization that you were asking the wrong question altogether. There are so many times someone came to me with a problem with a system I understood that they did not and it was clear they were asking the wrong question. Frequently, I could tell them what question they were trying to ask because I knew things they didn't know they didn't know.
You may find Bloom's taxonomy interesting as a frame for thinking about how to achieve better understanding: https://en.wikipedia.org/wiki/Bloom%27s_taxonomy
Lots of good tactics here, but ultimately I think everything boils down to understand the problem better.
Your solution will only come from superior understanding of the problem, if there is one. It probably won't come from testing different solutions.
Almost all software starts with "what is the intput and what is the output" and then understanding how the input leads to the output. If you don't know the input and the output to your system, then you don't have understanding.
Even in your text, you have a "solution" based frame to your statement. Subordinating the problem to the solution rather than the solution to the problem puts so much focus on the solution that the problem itself gets defined in terms of the solution. I have seen feature developers play this out time and time and time again. When the problem gets redefined in terms of the solution, often the original problem remains resulting in increased complexity at little additional value.
So I would start by understanding that not being sure a problem has a solution means you don't understand the problem and its context enough to even be asking if it has a solution which means you should be asking "how do you better understand problems?" or "when do you stop investing in understanding problems?" or "what is a good time trade off between 'understanding problems' and 'directly handling business concerns'"?
When you understand the problem, you'll probably come to the realization that you were asking the wrong question altogether. There are so many times someone came to me with a problem with a system I understood that they did not and it was clear they were asking the wrong question. Frequently, I could tell them what question they were trying to ask because I knew things they didn't know they didn't know.
You may find Bloom's taxonomy interesting as a frame for thinking about how to achieve better understanding: https://en.wikipedia.org/wiki/Bloom%27s_taxonomy
An unsolvable problem in personal life is not a problem, but rather a condition. It is an experience that a person must spend time learning how to work with/through/around or how to accommodate.
If the challenge is something work related (i.e. I do a lot of math) then I timebox it. My journal, status report, and other docs will read upon failure: "I spent X hours reviewing Y problem. I assessed A, B, and C. Their shortcomings were Yada Yada Yada."
If the challenge is relationship related, it's dicier. More information would be needed.
If the challenge is something work related (i.e. I do a lot of math) then I timebox it. My journal, status report, and other docs will read upon failure: "I spent X hours reviewing Y problem. I assessed A, B, and C. Their shortcomings were Yada Yada Yada."
If the challenge is relationship related, it's dicier. More information would be needed.
Something is driving your anxiety. Pinpoint that, and a solution will probably present itself. It might be a general fear of failure, anxiousness about working on the wrong thing, more immediate concerns about the problem being necessary for a job and food, .... All of those have different solutions.
One slightly more generic tip is that your task isn't to to solve the maybe unsolvable thing; it's to do a number of sub-tasks, learn more about it, communicate expectations to those who care, still schedule other important bits of your life despite a workload of unknowable magnitude, work on any essential backup plans, and so on. You have intermediate thoughts and actions between where you are and where you want to be. Focus there, and the fact that you're working on solvable problems might help.
A tangentially related idea -- math students often mistakenly assume the point of the homework is to figure out how to get it right. Rather, it's an opportunity to think through all the ways you can get it wrong and ingrain those so that in the future the right path forward is obvious. Failing to quickly solve the problem is the express point of the activity, and approaching the homework from the perspective of figuring out how to learn from it rather that how to get full marks tends to overcome a lot of mental hurdles and give you that latter benefit as a side effect. You're similarly, at least in the very near term, not actually working on getting the final solution; you're working on learning about the problem.
One slightly more generic tip is that your task isn't to to solve the maybe unsolvable thing; it's to do a number of sub-tasks, learn more about it, communicate expectations to those who care, still schedule other important bits of your life despite a workload of unknowable magnitude, work on any essential backup plans, and so on. You have intermediate thoughts and actions between where you are and where you want to be. Focus there, and the fact that you're working on solvable problems might help.
A tangentially related idea -- math students often mistakenly assume the point of the homework is to figure out how to get it right. Rather, it's an opportunity to think through all the ways you can get it wrong and ingrain those so that in the future the right path forward is obvious. Failing to quickly solve the problem is the express point of the activity, and approaching the homework from the perspective of figuring out how to learn from it rather that how to get full marks tends to overcome a lot of mental hurdles and give you that latter benefit as a side effect. You're similarly, at least in the very near term, not actually working on getting the final solution; you're working on learning about the problem.
> One slightly more generic tip is that your task isn't to to solve the maybe unsolvable thing; ... learn more about it, communicate expectations to those who care
I have to go through this at least once a week in my support role, knowing a problem does not have an obvious solution or that it might but we have yet to find it - and not being able to necessary say that to the user in those words - 'setting expectation' and 'communicate expectations to those who care.
Often I find that this is often 'good enough' for the short to medium term, get them back up and working as best you can, show that you understand the issue even if you don't know the solution.
I have to go through this at least once a week in my support role, knowing a problem does not have an obvious solution or that it might but we have yet to find it - and not being able to necessary say that to the user in those words - 'setting expectation' and 'communicate expectations to those who care.
Often I find that this is often 'good enough' for the short to medium term, get them back up and working as best you can, show that you understand the issue even if you don't know the solution.
I could argue that most tasks can be accomplished, but they may not be worth pursuing due to the cost and effort involved. It is astonishing to see how many global issues are political rather than technical, with groups often opposing or preventing the resolution or improvement of the problem.
Based on my life experience, I have learned that small actions can lead to significant outcomes, whereas starting with grand endeavors often leads to mental stagnation and lack of progress.
For instance, let's consider the challenge of addressing climate change and global warming. This is a complex issue composed of numerous interconnected problems. It becomes challenging to determine where to start, which aspect to tackle first, and from which angle to approach it. However, if we narrow our focus to a smaller component, such as plastics recycling, we gain more clarity. We can explore existing initiatives like the cleanup of the Pacific's floating garbage patch or the utilization of otherwise unrecyclable plastics in making blacktop pavement. This narrowed focus provides us with a direction to follow.
If we want to address emissions, a core aspect of combating global warming, the best technical approach would likely involve improving batteries that don't rely on rare metals. I've heard of advancements in saline-based and bacteria-grown batteries. By restricting the scope to batteries, we can identify specific areas to investigate. By avoiding broad and vague objectives, individuals can offer recommendations, share knowledge, and contribute effectively to the cause. If we remain with a large and vague goal it's very hard to make any progress at all.
Based on my life experience, I have learned that small actions can lead to significant outcomes, whereas starting with grand endeavors often leads to mental stagnation and lack of progress.
For instance, let's consider the challenge of addressing climate change and global warming. This is a complex issue composed of numerous interconnected problems. It becomes challenging to determine where to start, which aspect to tackle first, and from which angle to approach it. However, if we narrow our focus to a smaller component, such as plastics recycling, we gain more clarity. We can explore existing initiatives like the cleanup of the Pacific's floating garbage patch or the utilization of otherwise unrecyclable plastics in making blacktop pavement. This narrowed focus provides us with a direction to follow.
If we want to address emissions, a core aspect of combating global warming, the best technical approach would likely involve improving batteries that don't rely on rare metals. I've heard of advancements in saline-based and bacteria-grown batteries. By restricting the scope to batteries, we can identify specific areas to investigate. By avoiding broad and vague objectives, individuals can offer recommendations, share knowledge, and contribute effectively to the cause. If we remain with a large and vague goal it's very hard to make any progress at all.
[deleted]
I've found that the best solution for anxiety/paralysis in work is to come up with a small number of next steps.
You can always come up with a list of 3-5 next steps that seem pretty reasonable, and that you can reliably execute. That's the key, because if they're unreasonable or you're not sure of being able to execute them then they'll continue to produce anxiety/paralysis. But these steps aren't solving the whole problem, they're just progress towards it.
Any single step might be to find 5 relevant papers, to read the papers, to send e-mails to 5 people who might have ideas, to make a list of all the things you know you know and all the things you know you don't know, to spend 60 minutes writing free-association brainstorming, to test out 3 different solutions, and so forth. It's all going to depend on your situation.
But the point is that you can always come up with bite-sized tasks and then execute on them. And when you can't, the task is always to start going around to other people and asking them for help.
And of course some things don't have current solutions, which you need to accept to. Always look to identify the point where, after researching and reaching out to people and gathering a reasonable amount information, whether it makes sense to continue or to move on to something else entirely.
You can always come up with a list of 3-5 next steps that seem pretty reasonable, and that you can reliably execute. That's the key, because if they're unreasonable or you're not sure of being able to execute them then they'll continue to produce anxiety/paralysis. But these steps aren't solving the whole problem, they're just progress towards it.
Any single step might be to find 5 relevant papers, to read the papers, to send e-mails to 5 people who might have ideas, to make a list of all the things you know you know and all the things you know you don't know, to spend 60 minutes writing free-association brainstorming, to test out 3 different solutions, and so forth. It's all going to depend on your situation.
But the point is that you can always come up with bite-sized tasks and then execute on them. And when you can't, the task is always to start going around to other people and asking them for help.
And of course some things don't have current solutions, which you need to accept to. Always look to identify the point where, after researching and reaching out to people and gathering a reasonable amount information, whether it makes sense to continue or to move on to something else entirely.
The simple way to solve the problem, of course, is to not know it's unsolvable. Then work at it, tirelessly, until you come up with the solution.
The story from math is that of a student who fell asleep and was late going to his final. He walks in, sees three problems on the blackboard, and works frantically to solve them in the time he has left. Valiantly, he manages to solve all three, turns it in, and just hopes for a passing grade.
Later he gets a call from the professor who asks "do you know what you did?" The student's heart drops, thinking he's failed miserably. The professor continues: "You were only supposed to do the first two problems," the professor explained. "That last one was an example of an equation that mathematicians since Einstein have been trying to solve without success. I discussed it with the class before starting the test. And you just solved it!"
Note that this story is actually (approximately) true! George Dantzig, a UC Berkeley PhD candidate at the time, solved an unsolved problem in math as homework, and the plot was later used in Good Will Hunting. Dantzig was later awarded National Medal of Science by President Gerald Ford.
https://www.snopes.com/fact-check/the-unsolvable-math-proble...
The story from math is that of a student who fell asleep and was late going to his final. He walks in, sees three problems on the blackboard, and works frantically to solve them in the time he has left. Valiantly, he manages to solve all three, turns it in, and just hopes for a passing grade.
Later he gets a call from the professor who asks "do you know what you did?" The student's heart drops, thinking he's failed miserably. The professor continues: "You were only supposed to do the first two problems," the professor explained. "That last one was an example of an equation that mathematicians since Einstein have been trying to solve without success. I discussed it with the class before starting the test. And you just solved it!"
Note that this story is actually (approximately) true! George Dantzig, a UC Berkeley PhD candidate at the time, solved an unsolved problem in math as homework, and the plot was later used in Good Will Hunting. Dantzig was later awarded National Medal of Science by President Gerald Ford.
https://www.snopes.com/fact-check/the-unsolvable-math-proble...
The best part of that story is this quote from Dantzig himself, IMO:
> A year later, when I began to worry about a thesis topic, Neyman just shrugged and told me to wrap the two problems in a binder and he would accept them as my thesis.
The "two problems" being referenced here are, of course, the two unsolved problems he solved as homework.
> A year later, when I began to worry about a thesis topic, Neyman just shrugged and told me to wrap the two problems in a binder and he would accept them as my thesis.
The "two problems" being referenced here are, of course, the two unsolved problems he solved as homework.
A classic guide is How to Solve it, from Polya
https://en.wikipedia.org/wiki/How_to_Solve_It
https://en.wikipedia.org/wiki/How_to_Solve_It
I have just started to read another his book and I'm sure that George Polya's ideas is the best source for topicstarter. The second-best I can recommend is TRIZ:
https://en.wikipedia.org/wiki/TRIZ
https://en.wikipedia.org/wiki/TRIZ
I know what you're talking about. For me, it was a show-stopper. I felt destroyed. Yet, here I am! That's the most important thing for you right now-- keep the problem in perspective. Hard times pass. This is the best news I can give you.
- Not all problems need to be solved.
- People can relate to your struggle and may be willing to share time with you to think about your problems (me included) if it would help
- Take a break. Go outdoors, into nature, for an extended period of time. Focus on the present, not your work.
- Not all problems need to be solved.
- People can relate to your struggle and may be willing to share time with you to think about your problems (me included) if it would help
- Take a break. Go outdoors, into nature, for an extended period of time. Focus on the present, not your work.
Just accept that in many scenarios all you can hope for is to get close to a solution.
Depending on the domain you work in, close may be perfectly ok.
Then walk away knowing you did your best.
This sounds easier than it actually is, particularly if you have an obsessive compulsive personality.
It took me years to "let go".
What worked for me:
1) Experience. I am much better at prioritizing tasks today than when I was younger and I can focus on things that matter instead of going into rabbit holes with stuff that doesn't.
2) Running every day at lunch break and avoiding big meals. I believe my mind works better when I return to my desk, I'm awake and energized. I have a small snack usually fruits but definitely no carbs or refined sugar.
3) Having a "manual" hobbie. I got into film photography and directed my obsession to it. There's something about the almost totally manual process of film photography that has a calming effect on me. I know people that described a similar effect with painting, analog music production and writing.
4) Walk away. When I'm working on an issue and get stuck I say F this and leave, then come back to it later that day or even days later. It's almost like it is a totally different problem and usually appears much easier when you see it after a break. Call it a day and go home, go for a walk or work on other stuff.
Depending on the domain you work in, close may be perfectly ok.
Then walk away knowing you did your best.
This sounds easier than it actually is, particularly if you have an obsessive compulsive personality.
It took me years to "let go".
What worked for me:
1) Experience. I am much better at prioritizing tasks today than when I was younger and I can focus on things that matter instead of going into rabbit holes with stuff that doesn't.
2) Running every day at lunch break and avoiding big meals. I believe my mind works better when I return to my desk, I'm awake and energized. I have a small snack usually fruits but definitely no carbs or refined sugar.
3) Having a "manual" hobbie. I got into film photography and directed my obsession to it. There's something about the almost totally manual process of film photography that has a calming effect on me. I know people that described a similar effect with painting, analog music production and writing.
4) Walk away. When I'm working on an issue and get stuck I say F this and leave, then come back to it later that day or even days later. It's almost like it is a totally different problem and usually appears much easier when you see it after a break. Call it a day and go home, go for a walk or work on other stuff.
Definitely know what you mean. I often end up feeling anxious about things I’m working on, feeling like they might not matter even if I solve them, or that I might not be able to solve them.
So, at least for me, there are 2 sources of this anxiety:
1. Does it even make sense to solve this? - Best thing is to try to validate the problem. Find people to talk to, get feedback. Or think about alternatives and constraints. Every now and then, I start feeling like “what I work on makes no sense, I should just abandon it”. Only to then analyse it, walk through the different options and decisions and realise I’ve done this decision making exercise previously when starting the project and I still agree with the outcome. But I guess it’s good to recheck the assumptions every now and then.
2. Can I solve the problem? - I often get discouraged when I realise things are a lot more difficult than expected and I start to doubt myself. Usually results in me procrastinating and taking a break. Things that help me are: taking a deep breath, doing some searching for similar problems, breaking the problem down into smaller tasks, finding a smaller, yet similar problem to address first. Again, talking to people and reaching out for help can be great. People are often happy and flattered when you come to them with well-formed questions and acknowledge them as someone you consider an expert in a field.
Hope this helps. In general, when working on very hard problems that might not be solvable, try and break them down, find the riskiest piece that seems solvable, and try to tackle it. If you fail, at least you fail fast and you can rinse and repeat :)
So, at least for me, there are 2 sources of this anxiety:
1. Does it even make sense to solve this? - Best thing is to try to validate the problem. Find people to talk to, get feedback. Or think about alternatives and constraints. Every now and then, I start feeling like “what I work on makes no sense, I should just abandon it”. Only to then analyse it, walk through the different options and decisions and realise I’ve done this decision making exercise previously when starting the project and I still agree with the outcome. But I guess it’s good to recheck the assumptions every now and then.
2. Can I solve the problem? - I often get discouraged when I realise things are a lot more difficult than expected and I start to doubt myself. Usually results in me procrastinating and taking a break. Things that help me are: taking a deep breath, doing some searching for similar problems, breaking the problem down into smaller tasks, finding a smaller, yet similar problem to address first. Again, talking to people and reaching out for help can be great. People are often happy and flattered when you come to them with well-formed questions and acknowledge them as someone you consider an expert in a field.
Hope this helps. In general, when working on very hard problems that might not be solvable, try and break them down, find the riskiest piece that seems solvable, and try to tackle it. If you fail, at least you fail fast and you can rinse and repeat :)
One exercise that I find helpful is...
Finding what the components of $Thing will do, where else they may be useful, and being critical on if they're needed.
Sometimes you find that it's not as big as anticipated, there's existing work, or the idea just doesn't hold water.
In general, it helps to break the problem down. Trying to bite the whole thing at once is overwhelming
Sometimes I find that if I do 20% of this, it fixes a much bigger problem than what I was even looking at
Finding what the components of $Thing will do, where else they may be useful, and being critical on if they're needed.
Sometimes you find that it's not as big as anticipated, there's existing work, or the idea just doesn't hold water.
In general, it helps to break the problem down. Trying to bite the whole thing at once is overwhelming
Sometimes I find that if I do 20% of this, it fixes a much bigger problem than what I was even looking at
This may only suit situations where there is a clear starting and ending point, but I tend to think of things in terms of graphs. I'm at point A, I want to get to point B. If I take both small steps forward from A and backwards from B, I end up with a smaller problem. If I can repeat this numerous times (whether practically or merely in terms of thinking things through) I can end up with a smaller problem to which a solution is more obvious.
Nearly for any subject I try to always think with two opposing voices: one that says there's a solution and one that says it's impossible, and I always explore both sides of the argument.
If at some point some position sounds more plausible than the other, I try to put into words why is that. Either I convince myself of some position, for example that it is impossible, or the hardness of getting convinced at one position encourages me that maybe it's the other position that's true.
Either way I don't have anxiety because either I'm confident in some direction and making progress, even if it is at proving something is impossible, or the doubt from failing to prove it is impossible makes me more confident it is possible.
Even if you're making progress at the direction you didn't prefer, you should still see it as progress, and if you're confident in one direction you see it as an indication that you can make progress.
Anyways, it's not enough to just hold a general feeling that something is impossible, you need to put it into words and see if it's truly convincing. When it's just general intuitive unspoken feeling you can't change it, but when you make it concrete, you can understand what's possible.
If at some point some position sounds more plausible than the other, I try to put into words why is that. Either I convince myself of some position, for example that it is impossible, or the hardness of getting convinced at one position encourages me that maybe it's the other position that's true.
Either way I don't have anxiety because either I'm confident in some direction and making progress, even if it is at proving something is impossible, or the doubt from failing to prove it is impossible makes me more confident it is possible.
Even if you're making progress at the direction you didn't prefer, you should still see it as progress, and if you're confident in one direction you see it as an indication that you can make progress.
Anyways, it's not enough to just hold a general feeling that something is impossible, you need to put it into words and see if it's truly convincing. When it's just general intuitive unspoken feeling you can't change it, but when you make it concrete, you can understand what's possible.
The two opposing voices is reminiscent of game theoretic proof methods. Terry Tao has a great answer on mathoverflow that puts it in personal terms about how he solves problems [0]:
> One specific mental image that I can communicate easily with collaborators, but not always to more general audiences, is to think of quantifiers in game theoretic terms. Do we need to show that for every epsilon there exists a delta? Then imagine that you have a bag of deltas in your hand, but you can wait until your opponent (or some malicious force of nature) produces an epsilon to bother you, at which point you can reach into your bag and find the right delta to deal with the problem. Somehow, anthropomorphising the "enemy" (as well as one's "allies") can focus one's thoughts quite well. This intuition also combines well with probabilistic methods, in which case in addition to you and the adversary, there is also a Random player who spits out mathematical quantities in a way that is neither maximally helpful nor maximally adverse to your cause, but just some randomly chosen quantity in between. The trick is then to harness this randomness to let you evade and confuse your adversary.
[0] https://mathoverflow.net/a/38882
> One specific mental image that I can communicate easily with collaborators, but not always to more general audiences, is to think of quantifiers in game theoretic terms. Do we need to show that for every epsilon there exists a delta? Then imagine that you have a bag of deltas in your hand, but you can wait until your opponent (or some malicious force of nature) produces an epsilon to bother you, at which point you can reach into your bag and find the right delta to deal with the problem. Somehow, anthropomorphising the "enemy" (as well as one's "allies") can focus one's thoughts quite well. This intuition also combines well with probabilistic methods, in which case in addition to you and the adversary, there is also a Random player who spits out mathematical quantities in a way that is neither maximally helpful nor maximally adverse to your cause, but just some randomly chosen quantity in between. The trick is then to harness this randomness to let you evade and confuse your adversary.
[0] https://mathoverflow.net/a/38882
I focus more on the outcome I would be okay with and my progression towards it over the problem. As I learn more, I can get a better sense of what the outcome should be.
If I'm approaching the problem first and not the solution, I try to classify if the problem is technical or organizational. If it's technical, I try to identify each of the barriers, and search endlessly for something that looks like it would get me a little further along in getting insight into how to solve the problem. It its organizational, I look at how I would restructure how I am approaching the problem or work or communicate with others in solving the problem.
Sometimes, I need to redefine the problem, scope of the problem, or what a successful outcome looks like. For example, I wanted to find a way to verify that the reports I was entering in PDF forms were being filled out properly. I spent forever trying to find a tool or program the PDF form to be verified. After a while (several months), I realized that verifying a spreadsheet would be a lot easier, and that I could generate the same report from the spreadsheet. Once that perception changed, I was able to tackle the problem I had: not being able to ensure that a procedure was filled out properly.
If I'm approaching the problem first and not the solution, I try to classify if the problem is technical or organizational. If it's technical, I try to identify each of the barriers, and search endlessly for something that looks like it would get me a little further along in getting insight into how to solve the problem. It its organizational, I look at how I would restructure how I am approaching the problem or work or communicate with others in solving the problem.
Sometimes, I need to redefine the problem, scope of the problem, or what a successful outcome looks like. For example, I wanted to find a way to verify that the reports I was entering in PDF forms were being filled out properly. I spent forever trying to find a tool or program the PDF form to be verified. After a while (several months), I realized that verifying a spreadsheet would be a lot easier, and that I could generate the same report from the spreadsheet. Once that perception changed, I was able to tackle the problem I had: not being able to ensure that a procedure was filled out properly.
It depends on the problem. There are different approaches that I like to try:
- break the problem up into smaller parts which are easier to grasp, plus some framework to combine the individual parts. If this is possible, you might be able to solve all easier parts and thus the mother problem
- start with a problem that looks like the original one but which is easier. Once you have been able to solve it, you can clean up space in your head to focus on the step from the easy problem towards the final one. Also, during the process you have probably gained some experience and understanding of the mother problem
- just start with the problem without expecting to solve it. Then do something else. E.g. have a good night sleep, do some household chore etc. When you're not focusing on the problem, your brains continue to work on it so when you look at the problem again the next day, it can well be that your are able to link parts together that you were overlooking the day before
- talk about the problem with someone else. It could be of course that the other person has some hints that you didn't think about. But the main thing is that while talking and explaining the problem, you often get a better understanding of the problem yourself as well.
- break the problem up into smaller parts which are easier to grasp, plus some framework to combine the individual parts. If this is possible, you might be able to solve all easier parts and thus the mother problem
- start with a problem that looks like the original one but which is easier. Once you have been able to solve it, you can clean up space in your head to focus on the step from the easy problem towards the final one. Also, during the process you have probably gained some experience and understanding of the mother problem
- just start with the problem without expecting to solve it. Then do something else. E.g. have a good night sleep, do some household chore etc. When you're not focusing on the problem, your brains continue to work on it so when you look at the problem again the next day, it can well be that your are able to link parts together that you were overlooking the day before
- talk about the problem with someone else. It could be of course that the other person has some hints that you didn't think about. But the main thing is that while talking and explaining the problem, you often get a better understanding of the problem yourself as well.
- Experiment as much as possible
- Keep your feedback loop as short as possible between reading and doing
- Visualize things (debug outputs, scratchpad of short lived diagrams, etc.)
- When possible, envision what the finish line looks like and go backwards from there
The last bullet is especially useful for building tools or APIs where you can start with using your tool / API as if it does exist exactly how you want it in a best case scenario and work backwards to implement the solution you want. Break down the steps along the way.
I think with time comes desensitization to that initial feeling. It never really goes away, but as you slowly solve more and more problems self assurance slowly overrides that anxiety. One thing that really helps me is to break down a problem and its abstractions. It's so much easier to start with the more "obvious" or higher level facts, and slowly chip your way into the center.
Change the representation. Work small, simple examples by hand until you see the pattern. Try to start at the smallest simplest Most reduced example of the problem you can come up with and work your way to the next one. Once you see a pattern, try to change the representation and how you write it down invent some language for the problem it’s important parts of relationships aspects or dynamics of the problem come up with some terms are some symbols to represent those parts.
Celebrate the small wins find enjoyment in the process and motivate yourself with peptalk’s imagining how great it will be when you solve it, or how close you are. get a feel for the problem and believe that it has a solution. I just keep trying to get there, enjoy the uncertainty and the excitement of discovery.
Remember, you may be the first person to tred these paths. So don’t forget to stop to take it all in once in a while. Even if you’re not The first person to do so, you’re doing the first time you’ve been there, so still stop to take it all in because it’s important to you, and that makes it important.
Celebrate the small wins find enjoyment in the process and motivate yourself with peptalk’s imagining how great it will be when you solve it, or how close you are. get a feel for the problem and believe that it has a solution. I just keep trying to get there, enjoy the uncertainty and the excitement of discovery.
Remember, you may be the first person to tred these paths. So don’t forget to stop to take it all in once in a while. Even if you’re not The first person to do so, you’re doing the first time you’ve been there, so still stop to take it all in because it’s important to you, and that makes it important.
Oh god, I describe grad research as a series of existential crises.
"Is this work worth anything?"
"Am I wrong?"
"Does this path just lead to a giant brick wall?"
I find there are a couple of tips that will keep you sane when tackling problems like this:
- Don't try to solve the problem at first. Spend more time than you think you should characterizing it, and describing how you'd know you'd reach a solution
- you will lose motivation and start to doubt yourself if you dont achieve small, conclusive milestones. Embark on a problem, set an ambitious deadline, and if you haven't hit your mark, divide it up again, start with the most doable segment and repeat.
- don't take shortcuts. This will only feed the doubt you already have. The biggest antidote for the existential crises I describe is trusting yourself. That becomes easier when you convince yourself of your methodology before you embark on a problem. Shortcuts lead to wobbly assumptions which undermine conclusiveness and just makes all your work feel like it rests on a foundation of sand and regrets.
"Is this work worth anything?"
"Am I wrong?"
"Does this path just lead to a giant brick wall?"
I find there are a couple of tips that will keep you sane when tackling problems like this:
- Don't try to solve the problem at first. Spend more time than you think you should characterizing it, and describing how you'd know you'd reach a solution
- you will lose motivation and start to doubt yourself if you dont achieve small, conclusive milestones. Embark on a problem, set an ambitious deadline, and if you haven't hit your mark, divide it up again, start with the most doable segment and repeat.
- don't take shortcuts. This will only feed the doubt you already have. The biggest antidote for the existential crises I describe is trusting yourself. That becomes easier when you convince yourself of your methodology before you embark on a problem. Shortcuts lead to wobbly assumptions which undermine conclusiveness and just makes all your work feel like it rests on a foundation of sand and regrets.
Pair with ChatGPT 4 to set up the project design.
I try to give as much context as possible. Stating goals and risks of the project. The more info you give it, the better it does at helping you. Around 500 words for an initial prompt gets good results.
Break down the project into parts and take on the most risky/uncertain part first.
As you work through the design, continue to refine the ChatGPT conversation by giving it more info. Keeping around 200 words is good. New articles can be summarized to get it up to speed on post 2021 info. Blog posts can also be chunked out and passed through to give it more info about the problem.
When the plan seems good enough, code through the steps to create the solution. Keeping things realistic and making some slow but steady progress is better than hammering it all out in a day. Discipline over passion.
I used it for a recent project and it completely exceeded my expectations. Not trying to hook onto the current AI bandwagon, it just brought me to solutions I would have never gotten to on my own.
I try to give as much context as possible. Stating goals and risks of the project. The more info you give it, the better it does at helping you. Around 500 words for an initial prompt gets good results.
Break down the project into parts and take on the most risky/uncertain part first.
As you work through the design, continue to refine the ChatGPT conversation by giving it more info. Keeping around 200 words is good. New articles can be summarized to get it up to speed on post 2021 info. Blog posts can also be chunked out and passed through to give it more info about the problem.
When the plan seems good enough, code through the steps to create the solution. Keeping things realistic and making some slow but steady progress is better than hammering it all out in a day. Discipline over passion.
I used it for a recent project and it completely exceeded my expectations. Not trying to hook onto the current AI bandwagon, it just brought me to solutions I would have never gotten to on my own.
It is the dark side of the engineer's mindset. The engineer thinks of neat problems and solutions.
However, that works only when you're dealing with certain types of contexts.
Many things can only be "improved", "managed" and "satisficed". For example, for many a politician, social stability, keeping the economic system going, maintaining peace can be considered "success".
Reading the biographies of people from different professions, and understanding their thinking seems to have helped me.
Another thing viewpoint is to see everything as a process, that just keeps running and evolving. Usually there is never a final solution to things. What's considered a "solved" problem pops up again in the future in some other form. Many problems are highly contextual, dependent on time/place. When the environment changes, the so-called "solution" no longer works.
Such flexibility of thought, that of questioning the limited paradigm of problem/solution may help as well.
However, that works only when you're dealing with certain types of contexts.
Many things can only be "improved", "managed" and "satisficed". For example, for many a politician, social stability, keeping the economic system going, maintaining peace can be considered "success".
Reading the biographies of people from different professions, and understanding their thinking seems to have helped me.
Another thing viewpoint is to see everything as a process, that just keeps running and evolving. Usually there is never a final solution to things. What's considered a "solved" problem pops up again in the future in some other form. Many problems are highly contextual, dependent on time/place. When the environment changes, the so-called "solution" no longer works.
Such flexibility of thought, that of questioning the limited paradigm of problem/solution may help as well.
That's an interesting way of thinking. Thank you. Do you mind sharing some biographies that influenced you?
I see multiple ways of what you are experiencing.
* There is no known solution or path to the solution
I usually just try to solve it with my current knowledge, as practical as i can (implementing it). Even if i cant solve it that way, i will have learned something and i can try again another time or using another approach. Looking for similar topics and approaches helps. Mind-Map thinking can help broadening the field and see adjacent topics which help finding a solution. Very often the knowledge from other fields help me solve something i could not without the broader knowledge.
* There is no known solution or path to the solution
* There might be none
* There might be one
* There might be multiple
* These might be approximations
* The solution will be so complex, it will not be sustainable
I usually just try to solve it with my current knowledge, as practical as i can (implementing it). Even if i cant solve it that way, i will have learned something and i can try again another time or using another approach. Looking for similar topics and approaches helps. Mind-Map thinking can help broadening the field and see adjacent topics which help finding a solution. Very often the knowledge from other fields help me solve something i could not without the broader knowledge.
Further option
* You might be bought-in, eg emotionally, to a situation that prevents solution
This is quite common in scientific research. The typical algorithm I follow is to reframe the problem in the language of different fields and see whether there is a more useful way of tackling it in that framework. There are always some leaks in the abstraction/translation but often by reframing the problem you find a good-enough solution.
A thing I've often fantasized about is some sort of mega-conference where top luminaries from every academic field get together and hammer out a global namespace of jargon, resolving all collisions so that no longer can a term mean eight different things in eight different fields.
Imagine the global boost in productivity and knowledge-sharing...
Imagine the global boost in productivity and knowledge-sharing...
I fantasize about this sometimes for scientific purposes. Part of the challenge is how to keep the barrier to entry low enough that people stay excited and creative, because the opposite is the world of regulated industries where your way of thinking is heavily influenced by the legal framework. Periodic synchronization helps and the best I have seen in person at the Gordon Research Conferences (GRC) since these tend to be narrow enough to have consensus but still broad enough to get a little bit of perspective
Most problems I tackle by breaking them down to the smallest possible problem and then re-build from there.
Failing fast at a smaller problem is a lot easier (which is what my last set of difficult problems ended up failing at) ... turns out applying a LUT to a camera stream on Android is a harder problem than expected if you want good performance.
Failing fast at a smaller problem is a lot easier (which is what my last set of difficult problems ended up failing at) ... turns out applying a LUT to a camera stream on Android is a harder problem than expected if you want good performance.
There are a number of ways you can approach any problem. A lot depends on the area you are talking about but you haven't provided anything to narrow the scope.
For example, Logic Mathematical Inference and a few other areas have formal methods which you can use to identify properties which can then be used towards or against proving something.
Computation for example has System's Properties that are required (such as determinism, time invariance, and step), and certain problems are derived from that which dictate some of the fundamental limits of computation.
When you know what the limits are, you know when you start pushing up against impossible problems.
Sometimes its useful to think of impossible problems as possible, and try to build a bridge, but these often require significant background knowledge and experience to avoid circular thinking of self reference.
For example, Logic Mathematical Inference and a few other areas have formal methods which you can use to identify properties which can then be used towards or against proving something.
Computation for example has System's Properties that are required (such as determinism, time invariance, and step), and certain problems are derived from that which dictate some of the fundamental limits of computation.
When you know what the limits are, you know when you start pushing up against impossible problems.
Sometimes its useful to think of impossible problems as possible, and try to build a bridge, but these often require significant background knowledge and experience to avoid circular thinking of self reference.
I look on things from a game theory perspective. Is worrying about the problem helping you to solve it? Usually the answer is no, but what does help instead? I find that bisecting the issues involved is a good start. It's practically impossible to control for many different variables all at once. And you can't control what you cannot measure, so adding better means of measurement is good too. Adding extra logging of information to your system for example.
Also sometimes the problem that needs resolution isn't what you think at the outset, so keep an open mind. Maybe your implicit assumptions are at fault so you may need to go back several steps to identify and validate those first.
Indeed, the 'solution' may be to do nothing at all but rather to think about the world in a new way. The 'problem' to be solved might then disappear like mist.
Also sometimes the problem that needs resolution isn't what you think at the outset, so keep an open mind. Maybe your implicit assumptions are at fault so you may need to go back several steps to identify and validate those first.
Indeed, the 'solution' may be to do nothing at all but rather to think about the world in a new way. The 'problem' to be solved might then disappear like mist.
Do you HAVE to tackle it?
Is it your idea, to tackle it or is someone else pushing for the problem to be fixed?
Is there anyone who knows for sure if it can be done or not?
Does it sound reasonable, within the known laws of physics/engineering that the problem is solvable?
Is that problem that unique that you can't just google it?
Make a list of all the possible attempts to solve it; ask all parties involved if they have other ideas or suggestions, and then time box each one of them.
Then start attempting each option.
If it takes you longer, document what you tried and either move on or ask someone else if it's worth it to continue that approach.
For the anxiety:
Why are you anxious about? In the worst case scenario where the problem is not solvable, what does that mean to you? Are you anxious you won't do it fast enough? Or you won't do it in the best way? Or that you're not capable to do it and someone else is capable of doing it?
Is it your idea, to tackle it or is someone else pushing for the problem to be fixed?
Is there anyone who knows for sure if it can be done or not?
Does it sound reasonable, within the known laws of physics/engineering that the problem is solvable?
Is that problem that unique that you can't just google it?
Make a list of all the possible attempts to solve it; ask all parties involved if they have other ideas or suggestions, and then time box each one of them.
Then start attempting each option.
If it takes you longer, document what you tried and either move on or ask someone else if it's worth it to continue that approach.
For the anxiety:
Why are you anxious about? In the worst case scenario where the problem is not solvable, what does that mean to you? Are you anxious you won't do it fast enough? Or you won't do it in the best way? Or that you're not capable to do it and someone else is capable of doing it?
These are important questions.
So many wild goose chases come from never asking "Who actually want this, and why do they think they need it?"
So many wild goose chases come from never asking "Who actually want this, and why do they think they need it?"
[deleted]
What kind if problem?
- Mathematics/physics textbook problem: can you prove it has no solution? E.g. you can start by assuming it has and then find a contradiction. That also works for finding a solution.
- Programming: can it be done by a series of pen-and-paper operations? If yes, then it has solution and you must keep on trying. Can you find an equivalence between your problem and and a well known problem? This last point is true also for math/physics.
On equating your problem with a well-known one: there are not so many new things under the sun. Try to formalize the problem statement, then you might come to conclusions of the form:
- This is equivalent to solving the traveling salesman (bad for you) - This is essentially a system of linear equations (good)
When you do this you can take advantage of all existing literature, i.e. the knowledge of smarter people.
- Mathematics/physics textbook problem: can you prove it has no solution? E.g. you can start by assuming it has and then find a contradiction. That also works for finding a solution.
- Programming: can it be done by a series of pen-and-paper operations? If yes, then it has solution and you must keep on trying. Can you find an equivalence between your problem and and a well known problem? This last point is true also for math/physics.
On equating your problem with a well-known one: there are not so many new things under the sun. Try to formalize the problem statement, then you might come to conclusions of the form:
- This is equivalent to solving the traveling salesman (bad for you) - This is essentially a system of linear equations (good)
When you do this you can take advantage of all existing literature, i.e. the knowledge of smarter people.
First and foremost - suspend any disbelief about whether you can solve it. This is hugely important. If it has a solution and you are going to be able to solve it, then it needs your full undivided attention and focus and motivation in order to make the best progress. The possibility that it doesn't have a solution and/or you aren't able to find one is the pathological case - you should only be concerned about the successful strategy and not be spending time on the pathological one. Secondly, be aware that difficult problems which haven't already been solved are unsolved for a reason. Be willing to be patient and dedicated to a long term investment of energy and time on your part to work on it
I have this quite often (most of the stuff my company has not been done before, or at least not properly/usable; for good or for bad) and I get this paralysis sometimes (more in the past than I do now). My mental way of solving it, is to imagine a product that does something similar to the actual 'perfect' solution, but does it far more crappy. Simply put; I make a shitty prototype first (with the thought that the perfect thing will come 'some time' after) that is not perfect. Often will transpire that this shitty prototype is as close to what we set out to do as we can get without unrealistic spending and effort and we leave it at that (we refactor it before launch though). This takes away my anxiety about 'finally doing the perfect' thing.
Divide and Conquer.
Break your problem into a set of smaller problems. Then partition this set of problems into tractable and intractable subsets. The definition of tractable and intractable can change over time, but must be precise and computable.
Start addressing the tractable problems and show progress. For intractable problems, research effort is required and the costs must justify benefits.
Let us say 60% of your customers are facing issue type A, 30% are facing issue type B, 10% are facing issue type C. Issue type B has a workaround/known solution whereas issue type A and C don't have obvious solutions. Instead of getting stuck with working on issue type A, you and your customers will be better off if you worked on and deployed the solution for issue type B. That is better than the status quo.
Break your problem into a set of smaller problems. Then partition this set of problems into tractable and intractable subsets. The definition of tractable and intractable can change over time, but must be precise and computable.
Start addressing the tractable problems and show progress. For intractable problems, research effort is required and the costs must justify benefits.
Let us say 60% of your customers are facing issue type A, 30% are facing issue type B, 10% are facing issue type C. Issue type B has a workaround/known solution whereas issue type A and C don't have obvious solutions. Instead of getting stuck with working on issue type A, you and your customers will be better off if you worked on and deployed the solution for issue type B. That is better than the status quo.
Team control and coordination is incredibly important when you don’t know if you can solve a problem. Inherently, if you don’t know if there’s a solution for your problem, your research will be very wide and your teams ability to adapt quickly to new directions is paramount.
As the leader, you may need to shut down directions of inquiry often. Every situation where “I wish I had pulled the plug on that sooner…” wasted time and effort.
Then you have to be willing to pull the plug for good. Whether you’re an employee or an external advisor, it’s always difficult to justify time and expense with no solution, but wasted/higher time and expense is worse.
This also applies to the individual, you still have to fight your own internal desire to push something to conclusion.
As the leader, you may need to shut down directions of inquiry often. Every situation where “I wish I had pulled the plug on that sooner…” wasted time and effort.
Then you have to be willing to pull the plug for good. Whether you’re an employee or an external advisor, it’s always difficult to justify time and expense with no solution, but wasted/higher time and expense is worse.
This also applies to the individual, you still have to fight your own internal desire to push something to conclusion.
I work in fields where we usually deal with projects where we are not sure has a solution. This includes computer security, system programming, and research. I am in a management role now but have worked in all positions of the company, many times simultaneuously.
First, it is important that your customer and/or team know that you are in that problem context. If you are alone there is a big issue and you should communicate this immediately. There is a saying (Knuth?) about how to differente between an exercise or research problem: if you can solve it it is an exercise, if you can't it is a research problem.
Anxiety probably comes from a mismatch between the complexity of the problem you have in front of you and the expectations others have on you.
First, it is important that your customer and/or team know that you are in that problem context. If you are alone there is a big issue and you should communicate this immediately. There is a saying (Knuth?) about how to differente between an exercise or research problem: if you can solve it it is an exercise, if you can't it is a research problem.
Anxiety probably comes from a mismatch between the complexity of the problem you have in front of you and the expectations others have on you.
I take a huge project that could takes weeks or months. And try to break down the task into 3 pieces for delivery.
Then I take each piece to work on and daily make a list of 3-5 tasks i feel I could complete in the day towards the overall goal. Sometimes task #1 is super simple, eg put name on top of paperwork today, or cleanup stale files, etc. But the goal is to always be able to say I completed a task towards the overall current piece of the whole project I'm building.
The stress is still there but everyday I can say ... See how far you've come, little by little it's coming together; the "process" is reassuring.
Keep reviewing the overall project plan, and biting off little pieces. It helps to manage the anxiety in this way.
Then I take each piece to work on and daily make a list of 3-5 tasks i feel I could complete in the day towards the overall goal. Sometimes task #1 is super simple, eg put name on top of paperwork today, or cleanup stale files, etc. But the goal is to always be able to say I completed a task towards the overall current piece of the whole project I'm building.
The stress is still there but everyday I can say ... See how far you've come, little by little it's coming together; the "process" is reassuring.
Keep reviewing the overall project plan, and biting off little pieces. It helps to manage the anxiety in this way.
This situation used to be called neurosis. A person faces some hard problem, death would be the paradigmatic one, and unable to come up with a real solution, adopts some stereotyted behaviour of avoidance or denial to cope.
If it is unsolvable, the only strategy is to learn to live with it. Or of it's "just" very hard, you need to find dignity in fighting and losing. You might even win, but it's more difficult if you're anxious.
Also consider that concern and guilt are weapons that some organizations use to involve people in their agendas. Persons that struggle to have a minimally functional life feel as if it's their responsability to save the world. Sometimes the answer is: fuck the planet and live a little.
If it is unsolvable, the only strategy is to learn to live with it. Or of it's "just" very hard, you need to find dignity in fighting and losing. You might even win, but it's more difficult if you're anxious.
Also consider that concern and guilt are weapons that some organizations use to involve people in their agendas. Persons that struggle to have a minimally functional life feel as if it's their responsability to save the world. Sometimes the answer is: fuck the planet and live a little.
In the face of such problems, it is difficult to draw conclusions directly, and the solution to some things is often found in the process of doing.
If you're going to solve something like this, then a better way is to find the right way to do it first, but often this correct way is only discovered in the process of doing it.
Haha, it's like a paradox, if you don't do this then you can't find a solution, and as a result, this thing can't be done, and if you do it, then you will face an uncertainty.
So, you need to keep trying and trying and making mistakes, and then make adjustments, get closer to the unknown right direction, and finding the right direction is not a one-time achievement.
If you're going to solve something like this, then a better way is to find the right way to do it first, but often this correct way is only discovered in the process of doing it.
Haha, it's like a paradox, if you don't do this then you can't find a solution, and as a result, this thing can't be done, and if you do it, then you will face an uncertainty.
So, you need to keep trying and trying and making mistakes, and then make adjustments, get closer to the unknown right direction, and finding the right direction is not a one-time achievement.
Divide and conquer.
Break the problem up into pieces and solve the pieces.
Break the problem up into pieces and solve the pieces.
Wow, I can’t believe this is at the bottom of the comments. This is exactly the approach regardless of whether the problem is at work or at home.
Problems: Analyze, break down, examine from all angles and add details. Then try solving.
I spent my career trying to teach this exact concept to my engineering teams. I agree with you 100%.
Problems: Analyze, break down, examine from all angles and add details. Then try solving.
I spent my career trying to teach this exact concept to my engineering teams. I agree with you 100%.
And if you think it's atomized enough, yet you still have problems then it's not atomized enough. You might have to get deep into first premises and axiom-land.
I have spent the past couple of years solving problems that, evidently, no one else is trying to solve (or else everyone else finds it easy to solve).
The evidence is that I spend a lot of time searching for error messages of which I am apparently the first recipient in the world.
I’m working with systems that are opaque and hostile to developers. I’m using platforms in ways that are very removed from the mainstream, provided by enormous companies.
I know a solution exists but what I don’t know is if a business viable solution exists.
The way I deal with it is to focus at each stage on what I do know, what I need to find out, and whether it is worth trying.
The “worth trying” part is often subjective but metrics such as revenue can help in the decision making process.
The evidence is that I spend a lot of time searching for error messages of which I am apparently the first recipient in the world.
I’m working with systems that are opaque and hostile to developers. I’m using platforms in ways that are very removed from the mainstream, provided by enormous companies.
I know a solution exists but what I don’t know is if a business viable solution exists.
The way I deal with it is to focus at each stage on what I do know, what I need to find out, and whether it is worth trying.
The “worth trying” part is often subjective but metrics such as revenue can help in the decision making process.
Legendary maker Jimmy DiResta once mentioned in the Making It podcast (which I can recommend by the way!) how his process was with an example that I really liked.
He had gotten some new heavy machine, some 400 pound stuff, that he needed to put down in the basement.
He had no idea how he'd get it down the stairs, but he started pushing it closer, since that got the problem closer to a solution. He'd figure the stairs later, once the machine was on the edge.
It's simple, but I found it insightful.
When I'm faced with a problem and am not sure how to solve it or if it's actually solvable, I think of how a solution would look like and start making small steps in that direction, hoping to get elucidated later on :-)
He had gotten some new heavy machine, some 400 pound stuff, that he needed to put down in the basement.
He had no idea how he'd get it down the stairs, but he started pushing it closer, since that got the problem closer to a solution. He'd figure the stairs later, once the machine was on the edge.
It's simple, but I found it insightful.
When I'm faced with a problem and am not sure how to solve it or if it's actually solvable, I think of how a solution would look like and start making small steps in that direction, hoping to get elucidated later on :-)
You will be sometimes wrong and sometimes right. So don’t punish yourself when you are not sure.
Always be transparent over how confident you are in your analysis and what will be the consequences if you are wrong.
Try to always focus on the practical implications of what you are analyzing.
Always keep in mind what are you trying to prove/disprove. If you to dismiss a business idea for example, you do not need to analyze every possible aspect of you can rule out the opportunity based on napkin calculations.
I worked with pricing ML models for some time, and if a problem has a 40% irreducible error rate you shouldn’t account for uncertainties that sum up to 0.3% of it.
Try to find similar but easier problems.
Analysis is just an activity, and is not the whole job.
Always be transparent over how confident you are in your analysis and what will be the consequences if you are wrong.
Try to always focus on the practical implications of what you are analyzing.
Always keep in mind what are you trying to prove/disprove. If you to dismiss a business idea for example, you do not need to analyze every possible aspect of you can rule out the opportunity based on napkin calculations.
I worked with pricing ML models for some time, and if a problem has a 40% irreducible error rate you shouldn’t account for uncertainties that sum up to 0.3% of it.
Try to find similar but easier problems.
Analysis is just an activity, and is not the whole job.
Context matters. Why you're there matters.
In a business context, there's always an itching problem or a potential value creation and you can frame it in those terms and timebox it (drop it after X time working on it and write about the results of your tests).
As much as possible, you should have a plan B that is workable even if suboptimal.
On the "problem solving itself". Just break down the problem as much as possible to ensure you're slowly removing risk and learning early if there's a problem. That helps with the paralysis since tasks should feel tryable.
If you can't even think about the problem then you clearly don't know enough about the domain and some separate theory and practice should help.
In a business context, there's always an itching problem or a potential value creation and you can frame it in those terms and timebox it (drop it after X time working on it and write about the results of your tests).
As much as possible, you should have a plan B that is workable even if suboptimal.
On the "problem solving itself". Just break down the problem as much as possible to ensure you're slowly removing risk and learning early if there's a problem. That helps with the paralysis since tasks should feel tryable.
If you can't even think about the problem then you clearly don't know enough about the domain and some separate theory and practice should help.
[deleted]
Not all problems are solvable, not all are worth solving. Not all need to solved fully and not all of those even need to be done automatically.
You often don't need to solve the problem to provide some business benefit.
So. Can you break it down at all? Is there some part of it that has a business benefit?
Can you do it manually, what's the trade-off of time and cost? Can you document it to make it a manual task for a lower level employee?
If not worth doing manually, often it's good to do some of it manually a few times so you get an idea of what's the same / different for different situations.
Can you automate the same parts?
Can you find a way to deal with some cases where it differs?
Can you now solve any of the remaining pieces?
This process can often be done with manual scripts, I forget the right term, where you create a script that prints out what the user should do and asks them to press enter when they've done it. You can then automate parts of it over time.
This kind of process helps have a useful output at all stages whenever a higher priority problem comes along. It's also easier to come back to it in a few months.
And if not, if you can't even do it manually it pushes it more towards unsolveable.
For problems that don't fit this kind of process, I often leave the problem to settle in my mind and talk it through while walking somewhere. Explain to imaginary people what we're doing what we can't do etc. Kind of like rubber duck debugging.
Prototypes, just trying can be good.
Most of all if you can, get an idea of the value of solving it and the value of your time. That helps put an upper bound on how long you should ever spend on it. A problem that causes a £10/mo extra aws charge a month is fundamentally not worth solving for almost any developer salary.
Finally, it's ok. It's ok if the problem can't be solved. It's ok if it's not worth it. It's ok if someone else sees the answer easily.
You often don't need to solve the problem to provide some business benefit.
So. Can you break it down at all? Is there some part of it that has a business benefit?
Can you do it manually, what's the trade-off of time and cost? Can you document it to make it a manual task for a lower level employee?
If not worth doing manually, often it's good to do some of it manually a few times so you get an idea of what's the same / different for different situations.
Can you automate the same parts?
Can you find a way to deal with some cases where it differs?
Can you now solve any of the remaining pieces?
This process can often be done with manual scripts, I forget the right term, where you create a script that prints out what the user should do and asks them to press enter when they've done it. You can then automate parts of it over time.
This kind of process helps have a useful output at all stages whenever a higher priority problem comes along. It's also easier to come back to it in a few months.
And if not, if you can't even do it manually it pushes it more towards unsolveable.
For problems that don't fit this kind of process, I often leave the problem to settle in my mind and talk it through while walking somewhere. Explain to imaginary people what we're doing what we can't do etc. Kind of like rubber duck debugging.
Prototypes, just trying can be good.
Most of all if you can, get an idea of the value of solving it and the value of your time. That helps put an upper bound on how long you should ever spend on it. A problem that causes a £10/mo extra aws charge a month is fundamentally not worth solving for almost any developer salary.
Finally, it's ok. It's ok if the problem can't be solved. It's ok if it's not worth it. It's ok if someone else sees the answer easily.
Come up with a list of potential approaches, order them by roughly how promising you think they are and then try them all in that order. Obviously limit this by how much time/energy you can offer it somehow.
When you're done, see if you can think of more approaches. If you can't, you probably can't solve this problem and it's time to call it a day.
But that's ok, because there's almost certainly some higher level goal you're aiming to achieve. So think about other ways that you could achieve that without solving your original problem. Maybe there's a way to solve a similar problem that has looser constraints which is easier.
When you're done, see if you can think of more approaches. If you can't, you probably can't solve this problem and it's time to call it a day.
But that's ok, because there's almost certainly some higher level goal you're aiming to achieve. So think about other ways that you could achieve that without solving your original problem. Maybe there's a way to solve a similar problem that has looser constraints which is easier.
It depends why you think there might not be a solution. If you have enough experience and have a gut feel then do something simple and quick to prove the gut feel or put your hand up and say that you don't know how to approach it.
If you are not experienced then your battle is with fear and depending on the cause, a good way of getting past that is to work with others who can encourage you when you do well and help you understand when you are going awry and why.
And ultimately, don't bullshit. Be honest about what you think and how you might approach something and if the boss isn't convinced, let them give the job to someone else.
If you are not experienced then your battle is with fear and depending on the cause, a good way of getting past that is to work with others who can encourage you when you do well and help you understand when you are going awry and why.
And ultimately, don't bullshit. Be honest about what you think and how you might approach something and if the boss isn't convinced, let them give the job to someone else.
In my own experience, alternating between bouts of self-doubt, hopelessness and despair followed by episodes of excitement, mania (the good kind) and progress from the fire of a new idea all seems normal and unavoidable to me. You just have to slog through the rough times and keep going. Don't listen to naysayers and don't give up but be sure to take breaks. Keep a journal of what hasn't worked in the past and summarize what you learned. Just writing it will put it in your subconscious for later use and occasionally you'll want to go back and revisit an idea that did not work.
If no "good" solutions come to mind, I try to think of the dumbest, most irresponsible ways I could solve the problem. A silly solution is still a solution.
From there, the game is to take that silly solution and work it toward something that's not so silly. It's not easy, but it gives you incremental successes that are vital when struggling with anxiety/paralysis. Also, deliberately looking for bad ideas takes off the pressure to have a good idea. (You may even find that the "bad" idea you were avoiding wasn't actually all that bad in the first place!)
From there, the game is to take that silly solution and work it toward something that's not so silly. It's not easy, but it gives you incremental successes that are vital when struggling with anxiety/paralysis. Also, deliberately looking for bad ideas takes off the pressure to have a good idea. (You may even find that the "bad" idea you were avoiding wasn't actually all that bad in the first place!)
I reference the 'Cutting the Gordian Knot' section in the Pragmatic Programmer[0] when dealing with tough problems.
Cutting the Gordian Knot [1] When solving impossible problems, ask yourself:
* Is there an easier way?
* Am I solving the right problem?
* Why is this a problem?
* What makes it hard?
* Do I have to do it this way?
* Does it have to be done at all?
[0] - https://en.wikipedia.org/wiki/The_Pragmatic_Programmer
[1] - https://github.com/HugoMatilla/The-Pragmatic-Programmer#cutt...
Cutting the Gordian Knot [1] When solving impossible problems, ask yourself:
* Is there an easier way?
* Am I solving the right problem?
* Why is this a problem?
* What makes it hard?
* Do I have to do it this way?
* Does it have to be done at all?
[0] - https://en.wikipedia.org/wiki/The_Pragmatic_Programmer
[1] - https://github.com/HugoMatilla/The-Pragmatic-Programmer#cutt...
I think you don't provide enough information for anybody to give you good advice.
That being said: Consider that sometimes there are problems where you yourself will be unable to judge the problem you're in, because you are in it. It can be a bit of a bootstrapping problem: If you were the person who could see the solution, you might not have ended up in the situation in the first place. So the logical solution can be to get outside help so advice by a trusted person or actual therapy.
Getting out of a hole can sometimes be done by clawing yourself out, but sometimes having someone throw you a rope is the smarter move.
That being said: Consider that sometimes there are problems where you yourself will be unable to judge the problem you're in, because you are in it. It can be a bit of a bootstrapping problem: If you were the person who could see the solution, you might not have ended up in the situation in the first place. So the logical solution can be to get outside help so advice by a trusted person or actual therapy.
Getting out of a hole can sometimes be done by clawing yourself out, but sometimes having someone throw you a rope is the smarter move.
Every problem has a solution.
The more relevant question is: At what cost?
For example, in layman terms we all consider factoring RSA2048 impossible. But, given all the computing power in the world or 10 years more on quantum research this becomes possible.
For sure you can split hairs about this all day, like adding cost constraints as part of the problem and formal mathematical proofs may be an exception here.
In the end, what really matters is, how much is it worth to solve the problem and how much is it worth to estimate this cost. Investigations to reach an accurate estimate also cost effort.
The more relevant question is: At what cost?
For example, in layman terms we all consider factoring RSA2048 impossible. But, given all the computing power in the world or 10 years more on quantum research this becomes possible.
For sure you can split hairs about this all day, like adding cost constraints as part of the problem and formal mathematical proofs may be an exception here.
In the end, what really matters is, how much is it worth to solve the problem and how much is it worth to estimate this cost. Investigations to reach an accurate estimate also cost effort.
i feel like this is just the research process. My solution is to just try something that seems relevant. Soon you will realise it may not work because X. but maybe Y will work. Then you keep going, learning more about the area, the data, whatever. Stop when it works, or you have convinced yourself it wont. This is one of the reasons I think research code is always so trash, rarely did the researchers know where it was going to go at the start. And thats ok. Refactoring for production only happens well into the future.
my advice is to do something without worrying that it might not be the right thing.
my advice is to do something without worrying that it might not be the right thing.
I tell my brain there is a solution, you will figure it out, you always do. I then walk away from it but periodically return. You need to force it back to your subconscious mind that is more in touch with asking the entire universe to help you manifest a solution.
This may sound like a lot of woo but it usually works. One particularly hard problem took me eighteen months to solve. I should note that I'm stuck on a real bad bad problem right now though, been at it for nine months. It's not technical, more spiritual. I did get a solution but I am not sure I can follow through with it.
This may sound like a lot of woo but it usually works. One particularly hard problem took me eighteen months to solve. I should note that I'm stuck on a real bad bad problem right now though, been at it for nine months. It's not technical, more spiritual. I did get a solution but I am not sure I can follow through with it.
Given a complex problem, if there is just one part of it that I fear may not be solvable, then I will look into that part first to figure out if a solution exists and only later into the remaining elements of the problem.
If instead the uncertainty comes just from the "size" of the problem, then I split it into smaller parts that are easier to face, and I look into each of them the bare minimum to be able to confirm that a solution exists. Once I am confident a solution exists for all of the smaller parts and therefore for the whole problem, then I start the actual work.
If instead the uncertainty comes just from the "size" of the problem, then I split it into smaller parts that are easier to face, and I look into each of them the bare minimum to be able to confirm that a solution exists. Once I am confident a solution exists for all of the smaller parts and therefore for the whole problem, then I start the actual work.
I have been working on my fair share of problems that may or may not have solutions. Two pieces of advice that helped me:
1. Choose your battles. There are important problems out there, but our time is limited. If it is important and in my domain, I will pick it up. 2. Is it developmental? Depends on what stage you are in, but a couple of years ago I picked up lots of problems without solutions. It was a risky move, I wasted a ton of time, and the payoff wasn't obvious --- I definitely did some short-term damage. I am reaping some benefits now because of the skills I developed.
1. Choose your battles. There are important problems out there, but our time is limited. If it is important and in my domain, I will pick it up. 2. Is it developmental? Depends on what stage you are in, but a couple of years ago I picked up lots of problems without solutions. It was a risky move, I wasted a ton of time, and the payoff wasn't obvious --- I definitely did some short-term damage. I am reaping some benefits now because of the skills I developed.
I've been wanting to work on networking for multiplayer online browser games (but not io style more traditional game style) but it just seems like a ludicrously challenging problem for the kinds of games I want to make. Some are real-time, some are turn-based, all would definitely want an effective system for handling disconnects and latency and data persistence.
Trying to work through these is like far harder than any "hard" problem I've seen on leetcode. It actually makes me feel like sick to my stomach when I spend 12 hours on it without making a lot of progress
Trying to work through these is like far harder than any "hard" problem I've seen on leetcode. It actually makes me feel like sick to my stomach when I spend 12 hours on it without making a lot of progress
Yes, so essentially I try a few things, I try to find a naive and simplest solution and not ever worry about optimizations. If I fail in that case I try to find a simpler variation of the problem and solve that, if not I keep reducing the problem down. If the problem is reduced down I can explain it to someone else and ask them. Then or from that reduced problem I try and figure out what class of problems it belongs to. If you get anxiety you should ask around, and do what you are doing ask the internet using concise language.
I feel like it can't be understated how amazing 'asking the internet' is today. I regularly post questions on repos or SO where the software creators directly answer, with their intent. We have countless means of interacting with the ones that create the tools we work with (including HN!).
I know that there's some hairiness around maintainer manners and the drama expected in any large enough group of people, but our ability to communicate with our peers is really advanced compared to other disciplines.
I know that there's some hairiness around maintainer manners and the drama expected in any large enough group of people, but our ability to communicate with our peers is really advanced compared to other disciplines.
Don't take yourself so seriously if that creates anxiety and stress. If everyone around you expects you to be serious about it and that makes you feel uncomfortable in general, take that message from your brain seriously. It can become an everyday torture that takes you down to terrible places. People have praised and deride my solutions. Life goes on either way. I don't solve problems because I want to please people. I solve problems because I want to understand the causes and effects of everything. EVERYTHING! ;)
There may be no easy (general) solution, but there may be an acceptable solution in a particular case.
The halting problem is unsolvable, but you usually can say with certainty that a particular piece of code is not an infinite loop. The knapsack problem may take ridiculous time to solve precisely, but moat of the time you have little trouble packing your backpack or bag so that everything you need fits. Etc.
Often you only need to solve 20% of the problem to get 80% of the benefit. Make this step, and you will face further problems from a much better position.
The halting problem is unsolvable, but you usually can say with certainty that a particular piece of code is not an infinite loop. The knapsack problem may take ridiculous time to solve precisely, but moat of the time you have little trouble packing your backpack or bag so that everything you need fits. Etc.
Often you only need to solve 20% of the problem to get 80% of the benefit. Make this step, and you will face further problems from a much better position.
When I have an open-ended problem without an obvious solution, I often start by systematically questioning all of my assumptions. Why do I believe something is true? We often smuggle in beliefs about the problem that are of questionable provenance or are not actually true on careful inspection.
I find that this kind of first principles approach gives me a much better understanding of the nature of the problem. I may still not know how to solve it but it usually gives me insight into how to attack it most effectively.
I find that this kind of first principles approach gives me a much better understanding of the nature of the problem. I may still not know how to solve it but it usually gives me insight into how to attack it most effectively.
One approach: try to prove it doesn't have a solution. Would solving the problem entail things that obviously aren't true? Or does it entail things that probably aren't true, forcing you to dig farther about situations where they might be? Attempting to prove impossibility inverts the problem, and lets you act as critic on an argument that you want to be proved false.
This is a common approach to mathematical problems, but it's helped me break down more practical situations too.
This is a common approach to mathematical problems, but it's helped me break down more practical situations too.
I just had this conversation with my wife. She needs to expand her business into the nearest city and has been dragging on it for a couple of years. She admits she is not sure why she has not started. She is overwhelmed with the tasks needed to get it going and not sure where to start.
I suggested that the least important thing is How you start, and the most important thing is That you start. Starting the process, in any form, focuses you quickly on the most important things that need to get done first.
I suggested that the least important thing is How you start, and the most important thing is That you start. Starting the process, in any form, focuses you quickly on the most important things that need to get done first.
In research, you have this situation quite often.
Often you at least have some intuition or assumption on sth and then you just keep doing experiments to either confirm or falsify this. When sth does not work yet, you do experiments or analysis to better understand why it is failing. And then you try out more ideas to compensate those problems.
There is really no ending here, you can push it further and further as much as you like. You will get a better understanding along the way on what is possible and what is not.
Often you at least have some intuition or assumption on sth and then you just keep doing experiments to either confirm or falsify this. When sth does not work yet, you do experiments or analysis to better understand why it is failing. And then you try out more ideas to compensate those problems.
There is really no ending here, you can push it further and further as much as you like. You will get a better understanding along the way on what is possible and what is not.
First, make sure you understand the problem. Look at some particular examples of your problem where the solution is clear, understand the components of the problem. When your have a more or less complete image of the problem in your head, make a list of potential solutions/approaches. Think about the list that you made until one particular approach or solution appears to be the most fruitful and then pursue that. If/when you get stuck, go back to earlier steps of this process.
I was handed a problem during a job interview that was a "how do you get a computer to do this?" problem. Apparently I was the only one who attempted, and the general consensus is that no, you need a human being to do it.
So I sometimes divide problems into "Can a human do it or not?"
(Years later I would finally have the inklings of how to get a computer to do it. It would be quite complex but I think it might actually be feasible. A lot of subconscious introspection happened.)
So I sometimes divide problems into "Can a human do it or not?"
(Years later I would finally have the inklings of how to get a computer to do it. It would be quite complex but I think it might actually be feasible. A lot of subconscious introspection happened.)
Not knowing if something has a proper solution is a pretty normal state of affairs. Make a plan how to analyze it: for the next n days, what are you doing each day. Collect data x, run analysis y, book time on machine z, etc. Really write it down the activities and deliverables for each day for the next 2 to 3 weeks to start.
It does not mean there is no space for creativity - the plan will not last as new things come to light, but the plan means you move forward in a structured manner.
It does not mean there is no space for creativity - the plan will not last as new things come to light, but the plan means you move forward in a structured manner.
Without knowing the problem or context, I will take this on the basis of a "coding problem" where I may be tasked to create a piece of software/code/webapp to solve the needs. I take things a bit differently to most, I find the best way (for me) is to start tackling the problem with the tools I have, and then when I get stuck, see what I can do to fix it. I don't believe any problems I face have no solution, its just how long it takes/much it costs.
My approach: Divide and conquer
https://en.wikipedia.org/wiki/Divide_and_rule
https://en.wikipedia.org/wiki/Divide_and_rule
Can I make an observation? Most of life involves problems that may not have a solution or a demonstrably optimal solution. Most complex problems involve trade offs. Much of the skill in being an experienced person in one's field involves making the best possible trade offs toward a working solution. I like the other observations in this thread about failing fast too - often we find something that works, and we continue to iterate to improve.
To problems that can't be solved (like how can I get my wife back) I just imagine I have solved it, or my brain will just keep trying to solve the problem. For solvable problems i either brute force by moving the mountain one sand corn at at time, or keep iterating. When you have a grip on how much the problem will cost to solve you have to ask yourself it it's worth solving - or if you can just use a more crude existing solution.
I really enjoyed reading Fermat's Enigma[1] a book about the guy who solved Fermat's Last Theorem.
A memorable quote from the book:
"It’s fine to work on any problem, so long as it generates interesting mathematics along the way—even if you don’t solve it at the end of the day."
[1] https://www.amazon.com/Fermats-Enigma-Greatest-Mathematical-...
A memorable quote from the book:
"It’s fine to work on any problem, so long as it generates interesting mathematics along the way—even if you don’t solve it at the end of the day."
[1] https://www.amazon.com/Fermats-Enigma-Greatest-Mathematical-...
As a general answer I like to take sort of "5 whys" approach. Try to list all the contexts and constraints to the problem so that I can see potential conflicting/circular dependencies. This sometimes results in conflict identification something like "X is blocked by Y, but Y is blocked by Z". Maybe there is additional "Z is blocked by X", maybe it results in problem restatement to "solve Z".
I look for different ways of representing the problem domain or the desired outcome. This will improve your vocabulary/toolbox for reasoning about and subdividing them into more approachable units.
For some inspiration, I recommend watching this talk by Bret Victor https://www.youtube.com/watch?v=oUaOucZRlmE (Media for Thinking the Unthinkable)
For some inspiration, I recommend watching this talk by Bret Victor https://www.youtube.com/watch?v=oUaOucZRlmE (Media for Thinking the Unthinkable)
If you're lucky enough to be in a problem-solving situation where you have the space (mentally, physically, monetarily) to take this advice: When you can't find a breadcrumb in the middle, focus on the end and the start (in that order). Ignore the chance of a solution being impossible (but don't ignore evidence of a solution being impossible - use it to think about the end, the start, or the middle.)
Think about the problem deeply for several days. Take a 3 day break and then start working in the solution. Allow the associative cortex to do the work.
Read, think, and sleep. Problems in this category often need time in your brain to settle and clarify. The process is rewarding, and after you’ve tackled a few such problems (either proving them impossible or actually solving them) you gain confidence. A subset of unsolved or unsolvable problems might become long term projects that you can manage and prioritize (or eventually put aside) over the years.
Ask somebody from a different branch of work, who has very bad understanding of that problem.
They often have a bizzare idea somehow leading to the right track. They solve it in a completely different way or know that the problem does not matter to begin with.
Its like rubber duck programming with a smart duck.
I once had some weird data problem and complained about it to a sales guy. He called the customer to not enter certain things,and the problem was gone.
They often have a bizzare idea somehow leading to the right track. They solve it in a completely different way or know that the problem does not matter to begin with.
Its like rubber duck programming with a smart duck.
I once had some weird data problem and complained about it to a sales guy. He called the customer to not enter certain things,and the problem was gone.
> How do you battle against (self-inflicted) anxiety/paralysis when you are attempting to tackle a problem you are not sure has a solution?
Prove that a solution does not exist. This removes any form of doubt. It guarantees that no one else can find a solution either. Attempting to prove it may also help to more clearly understand what is required and what isn't, which may simplify the problem.
Prove that a solution does not exist. This removes any form of doubt. It guarantees that no one else can find a solution either. Attempting to prove it may also help to more clearly understand what is required and what isn't, which may simplify the problem.
Every problem has a solution, otherwise it would not be a problem.
If you get stuck, reframe the problem. Ask yourself why, what, where, whom.. in other words, figure out the true nature of the problem and why it exists. Is it really a problem or did your expectations for the result changed? There is no single answer here but keep asking questions and you will figure it out eventually.
Many problems provably don't have a solution. OP didn't specify a problem domain but in both maths and computing the possibility that what you are trying to achieve is mathematically impossible is a real one, and distinguishing the absence of a solution from your own lack of ability to find the solution that does exist is a real challenge with no easy answer.
Real-life problems always have a solution, but sometimes it isn't a technical one. Or a technical solutions exist, but is too expensive. To take your math example, e(x) = 0 have no solution, but in real life, taking x = -100 could be close enough for your purpose.
In real life problems often have multiple solutions and we don't like some of them, possibly the simplest ones. This is much like exceedingly expensive solutions to technical problems. In my experience customers faced with a costly solution reframe their problem and accept to do their business a different way. Their customers won't notice.
We lost the secret key for this bitcoin in the incinerator, can we get the key back?
That problem has neither a theoretical nor a practical solution.
That problem has neither a theoretical nor a practical solution.
There is a human one though: accept the loss and add processes to not repeat the mistake.
That what I was talking about in my first sentence, but my post wasn't clear, I admit. Thank you, it helped me clarify.
That what I was talking about in my first sentence, but my post wasn't clear, I admit. Thank you, it helped me clarify.
It's not clear that _every_ problem has a solution.
NP=P? There are many other famous math problems. No one found solutions yet and it is also unclear whether one is able to find a solution. And it's unclear if there is even a solution.
NP=P? There are many other famous math problems. No one found solutions yet and it is also unclear whether one is able to find a solution. And it's unclear if there is even a solution.
Often unsolvable problems can be decomposed into a set of solvable problems and hopefully just one or a few unsolvable problems.
By narrowing the unsolvable problem it can help to communicate the issue to others (e.g customers or management) or help to find a solution. If that fails it provides some solvable problems to work on. Often solutions, workarounds, or compromises become evident over time.
By narrowing the unsolvable problem it can help to communicate the issue to others (e.g customers or management) or help to find a solution. If that fails it provides some solvable problems to work on. Often solutions, workarounds, or compromises become evident over time.
If you're getting paralysis:
That means that you need to write down how you feel and you need to write down what you know about the problem.
From there:
1. Write down and solve/suggest for what you know on the edges of the problem
2. Extremely pinpoint what the problem is
3. Talk it out with the rubber duck/coworkers
4. Escalate the issue.
5. Design out the solution and leave question marks where you can't solve it.
6. Go out for a walk
From there:
1. Write down and solve/suggest for what you know on the edges of the problem
2. Extremely pinpoint what the problem is
3. Talk it out with the rubber duck/coworkers
4. Escalate the issue.
5. Design out the solution and leave question marks where you can't solve it.
6. Go out for a walk
I call my difficult experimental software projects names like Calm and Peace. Just looking at those words helps with any anxiety.
Are you working on P != NP? Jokes aside, this reminds me of working on a grad school thesis. I tend to find the biggest whiteboard (you can use any medium though) and lay out everything that is known and unknown about the problem. This helps avoid the sensation of "too massive", which can be anxiety or paralysis inducing.
Flip a coin (or roll a die). I'm not kidding. There has been research on this. It has two benefits. One, as you flip the coin, you'll find yourself rooting for a certain outcome -- that's what you should do, regardless of how the coin lands.
And in some cases where you truly have no idea, it helps push your brain into a choice.
And in some cases where you truly have no idea, it helps push your brain into a choice.
[deleted]
Some of the guidance given by the Cynefin framework helped me to make better decisions: https://www.everydaykanban.com/2013/09/29/understanding-the-...
Instead of thinking the problem is technical or scientific in nature, let's think about the same question for human relationship problems. Like how do we solve the inability of our elected, appointed or self-selected leaders to seek consensus or consider alternative points of view.
- Is this knowable or solvable or of problem-type solved? (e.g. information problem vs. work problem, do we need to do new science?)
- How do analogies for this resolve? e.g. what else is an n-part dynamic?
- What is a test for a solution?
- Do I want the general solution or just a specific result and is this actually a problem I need to solve?
- How do analogies for this resolve? e.g. what else is an n-part dynamic?
- What is a test for a solution?
- Do I want the general solution or just a specific result and is this actually a problem I need to solve?
Just mess around with stuff and techniques related to the problem. Also rubber duck debug the problem a lot. Describe it to people so that they can understand it. Listen closely to their questions even though they won't make a lot of sense to you at first glance.
Maybe re-formulate the problem from different perspectives, find the intention behind the problem, go "one step up". Find the space where "the problem" is only one of many ways to solve the intention. Doesn't always work, but sometimes.
This may not be helpful, but I think about it really hard for a long time :-) No joke, I've spent weeks thinking about a single problem whenever I had downtime. It works for me, but I don't think a lot of people want to think about something that much.
Sounds like the problem is not the problem, but the problem-solver. Many good suggestions here on solving toy problems, etc., but since you describe the real problem as mental (anxiety), do all the boring self care stuff too: #1 is get enough sleep.
Don't aim for the perfect/absolute solution. Sometimes a solution that solves a chunk of the problem (big or small) is just enough. Try to break the problem down, see where you can start, where you are going to need help and what will be for later.
I focus on the process.
Same as for problems I am certain have solutions.
For me, it's all about the work, not the fruit.
But the caveat, I focus on projects that are going to bear fruit because most problems that might bear fruit probably won't and I'd rather be productive than important.
YMMV and that's aOK.
Same as for problems I am certain have solutions.
For me, it's all about the work, not the fruit.
But the caveat, I focus on projects that are going to bear fruit because most problems that might bear fruit probably won't and I'd rather be productive than important.
YMMV and that's aOK.
Richard Hamming's talk, "You and Your Research" is applicable.
https://www.cs.virginia.edu/~robins/YouAndYourResearch.html
https://www.cs.virginia.edu/~robins/YouAndYourResearch.html
Stay stupid.
Do whatever seems to work or may work. Don't think about it. Just do it.
The midwit meme pretty much sums this entire problem up: https://www.swyx.io/simplicity-rush
Do whatever seems to work or may work. Don't think about it. Just do it.
The midwit meme pretty much sums this entire problem up: https://www.swyx.io/simplicity-rush
Knowing when to stop researching the problem and start colliding ideas is an art in itself.
It's always good to have an easier and more attractive backup problem in your back pocket to make your current problem jealous and your approach to it more playful.
It's always good to have an easier and more attractive backup problem in your back pocket to make your current problem jealous and your approach to it more playful.
When you find something that doesn't work, you should consider that a win. That means you can celebrate getting closer to the solution.
Reminds me of the Edison quote: "I have not failed. I've just found 10,000 ways that won't work."
Reminds me of the Edison quote: "I have not failed. I've just found 10,000 ways that won't work."
I continue as if there is a solution, but when I see an avenue suggesting it doesn't have a solution, I try to take that. Generally the info from trying to show the problem is unsolvable will help understand the problem better anyway.
I'm aware that I could fail to find a solution even if there is one, but I don't let that stop me exploring the space and learning as I do. The fact that there might not be a solution doesn't change anything.
Another good one: describe the problem and why it's so hard to someone else.
In my experience usually it means you are not solving the right problem - that problem you feel has "no solution" is like the outer appendages or surface of something else which is where the real problem is.
My advice would be push back the possibly unsolvable problem down the road, and deal with your axienty/paralysis first.
If your anxiety/paralysis is specifically tied to the problem, seek help from a mental health professional.
If your anxiety/paralysis is specifically tied to the problem, seek help from a mental health professional.
Usually it's a good idea to start with a simple base case with only a few moving parts/interactions such that you can handle it all in your head.
Only once you understand the base case well you complexify.
[deleted]
I think if there is no good solution, doing nothing might be the best thing. We generally forget this option, in many cases, there are always ways to side step and avoid the problem.
The mindset I use is to believe (without evidence) that a solution exists. That allows me to see solutions faster because I'm not fighting with myself.
Put yourself into the world where the solution exists.
Put yourself into the world where the solution exists.
Try to write down a semi-proof as to why it can't be solved.
I tend to find that assumptions/etc that get made in that proof can be attacked and something trackable can be found this way.
I tend to find that assumptions/etc that get made in that proof can be attacked and something trackable can be found this way.
Often the pivot to proving why the solution you looked for can't exist is just as rewarding. A lot of interesting would-be research lays in that avenue of investigation.
~"Part of an answer to a question lies in the definition of the question" (c) some famous physicist.
James Fadiman also conducted research on microdosing aiding problem solutions.
James Fadiman also conducted research on microdosing aiding problem solutions.
It’s very tough. For me, an important moment is realizing that these situations are rare and where the most growth can happen, which helps me push through sometimes.
Sometimes “there is no solution to this” is the solution to the problem. Often it may: “the solution to this is beyond us/me” - which is an equally suitable solution.
Think about the problem, actively and deeply. Then let it go and sleep on it.
Sometimes, when you can’t come to a solution, it’s best to let the solution come to you.
Sometimes, when you can’t come to a solution, it’s best to let the solution come to you.
I usually tackle the absolute hardest part first, hoping that I'll either fail fast on that, or it'll work and it will be downhill from there.
“Sometimes the best way to solve a problem is to stop participating in it.” Can be applied to various parts of the overall problem, or the entire thing.
Talking to others about the problem and or the situation will help both in a rubberducky way and as the saying goes a problem shared is a problem halved.
I look at it once, then ignore it and do something else for a few days. Revisiting the problem usually presents a couple of potential solutions.
Side question: How do you deal with micromanagement or incessant demands for estimations on a problem that you're not sure has a solution?
Feel like there's an "Ask HN: How do you approach a solution you are not sure has a problem?" counter thread opportunity here.
Every problem has a solution. When it is hard take a step back and return to the problem. Eventually you will find a solution or a quick hack
Taking long breaks and let the mind wander. Learn from failures. Understand the problem and possibly rethink what you are trying to achieve.
Step away from it, do something else. I find my mind will process it in the background. After a day or so, I may be ready to tackle it.
I assume that there is a solution, that the solution is already known, and that it's my approach that prevents me from seeing it.
I suppose it happens to everybody one day. I suppose it deals with self confidence and optimism. And confidence in the team…
Involving others could help. See if they have a solution, if not, maybe not to worry? I wonder what problem you speak of.
try a (or some) quick prototypes of the simplest possible form of the problem. Try and find if it maps to other, solved or proven unsolvable, problems. Split your problem into subproblems, see if those are solvable, if not completely, change course toward a more solvable, maybe similar, problem.
since you mentioned your open mind having difficulty, it sounds like you need to to improve the ability to tackle these issues more methodically and slowly plan out a strategy of attack instead of just trying things.
Assign it to a group of students new to the topic... without telling them that no known solution exists.
It may sound profound, but "Just do it!" Don't look too far ahead and take the first step.
Keep smashing my head against the wall until I make progress. Accept that it can take days to get an in.
If it doesn't have a solution, then find out why it doesn't and document that.
TDD. (I'm not religious about tdd, but this is the place where I use it always).
Perhaps you will find inspiration in Dave Snowden's framework "Cynefin"
Do you mean like a technical problem with code or a social/political issue?
Just start trying stuff. Learning happens through the trial and error process.
Set a deadline and try as hard as you can assuming it has a solution.
I find assuming that there IS a solution helps focus the mind.
Turn it into a problem where you are sure there’s a solution.
Reduce the problem to a simpler one you know how to solve.
“What is the smallest part of this problem can I solve”?
Relax the problem - remove one constraint and try again.
Deconstruct and aim to solve a lesser subproblem first.
There’s an interesting approach used by DARPA:
1. Prove its not impossible
2. Prove it’s possible
3. Do it
1. Prove its not impossible
2. Prove it’s possible
3. Do it
Ask gpt the answer, then correct its errors
Work consistently, and pray for strength, hope and wisdom. These are the best kinds of problems to work on, IMO, but they can get very difficult.
Assume it has a solution. Try that.
make your problem figuring out if the original one is solvable. more thinking, less typing
Solve it 80%
What's an example of one?
A thing I tell junior programmers is that their job isn't so much to solve problems, as to help figure out what the problem really is, and solve that. When somebody asks for a thing, don't go silent and go and build precisely that thing, and then come back weeks (or, heaven help us, months) later with that thing. Rather, talk to them! Hardly anybody knows what they want! Your job isn't to do precisely what they say they want, it's to help them figure out what they want! Iterate!
So, in that vein, both the solution *and the problem* are fair game for attack.
There is a wonderful, wonderful short essay by Michael Jackson (the other one) called "Brilliance" in his book "Software Requirements and Specifications". It is poignant, even painful, in the way that the the TV shows Silicon Valley or The Office are exquisitely painful. It is, alas, impossible to find unless you know exactly what you are looking for, but...
*Brilliance*, by Michael Jackson (from "Software Requirements and Specifications")
Some time ago I spent a week giving an in-house program design course at a manufacturing company in the mid-west of the United States. On the Friday afternoon it was all over. The DP Manager, who had arranged the course and was paying for it out of his budget, asked me into his office.
"What do you think?" he asked. He was asking me to tell him my impressions of his operation and his staff. "Pretty good," I said. "You've got some good people there." Program design courses are hard work; I was very tired; and staff evaluation consultancy is charged extra. Anyway, I knew he really wanted to tell me his own thoughts.
"What did you think of Fred?" he asked. "We all think Fred's brilliant." "He's very clever," I said. "He's not very enthusiastic about methods, but he knows a lot about programming." "Yes," said the DP Manager. He swiveled round in his chair to face a huge flowchart stuck to the wall: about five large sheets of line printer paper, maybe two hundred symbols, hundreds of connecting lines. "Fred did that. It's the build-up of gross pay for our weekly payroll. No one else except Fred understands it." His voice dropped to a reverent hush. "Fred tells me that he's not sure he understands it himself."
"Terrific," I mumbled respectfully. I go the picture clearly. Fred as Frankenstein, Fred the brilliant creator of the uncontrollable monster flowchart. That matched my own impression of Fred very well. "But what about Jane?" I said. "I thought Jane was very good. She picked up the program design ideas very fast."
"Yes," said the DP Manager. "Jane came to us with a great reputation. We thought she was going to be as brilliant as Fred. But she hasn't really proved herself yet. We've given her a few problems that we thought were going to be really tough, but when she finished it turned out they weren't really tough at all. Most of them turned out pretty simple. She hasn't really proved herself yet - if you see what I mean?"
I saw what he meant.
So, in that vein, both the solution *and the problem* are fair game for attack.
There is a wonderful, wonderful short essay by Michael Jackson (the other one) called "Brilliance" in his book "Software Requirements and Specifications". It is poignant, even painful, in the way that the the TV shows Silicon Valley or The Office are exquisitely painful. It is, alas, impossible to find unless you know exactly what you are looking for, but...
*Brilliance*, by Michael Jackson (from "Software Requirements and Specifications")
Some time ago I spent a week giving an in-house program design course at a manufacturing company in the mid-west of the United States. On the Friday afternoon it was all over. The DP Manager, who had arranged the course and was paying for it out of his budget, asked me into his office.
"What do you think?" he asked. He was asking me to tell him my impressions of his operation and his staff. "Pretty good," I said. "You've got some good people there." Program design courses are hard work; I was very tired; and staff evaluation consultancy is charged extra. Anyway, I knew he really wanted to tell me his own thoughts.
"What did you think of Fred?" he asked. "We all think Fred's brilliant." "He's very clever," I said. "He's not very enthusiastic about methods, but he knows a lot about programming." "Yes," said the DP Manager. He swiveled round in his chair to face a huge flowchart stuck to the wall: about five large sheets of line printer paper, maybe two hundred symbols, hundreds of connecting lines. "Fred did that. It's the build-up of gross pay for our weekly payroll. No one else except Fred understands it." His voice dropped to a reverent hush. "Fred tells me that he's not sure he understands it himself."
"Terrific," I mumbled respectfully. I go the picture clearly. Fred as Frankenstein, Fred the brilliant creator of the uncontrollable monster flowchart. That matched my own impression of Fred very well. "But what about Jane?" I said. "I thought Jane was very good. She picked up the program design ideas very fast."
"Yes," said the DP Manager. "Jane came to us with a great reputation. We thought she was going to be as brilliant as Fred. But she hasn't really proved herself yet. We've given her a few problems that we thought were going to be really tough, but when she finished it turned out they weren't really tough at all. Most of them turned out pretty simple. She hasn't really proved herself yet - if you see what I mean?"
I saw what he meant.
Use GPT
On problems that might not have a solution? Not much chance with the current (chat)gpt. There are many technical problems that have currently no solution and chatgpt will just keep repeating, over and over again, ways to attack the problem. Helpful, maybe, at first, but after 2-3 prompts, it'll just be more of the same as it cannot help solving it further.
Change the way you perceive the problem, see the solution as the opposite of the problem.
As a capitalist with resources, you hire someone to find a solution, dispose of them when they fail or profit from their labor when they succeed, and even then sometimes dispose of them if you don’t have more problems to solve.