Greedy algorithms are important for software engineers to know because they can offer quick and simple solutions to certain optimization problems (but note that they can also deliver less-than-optimal results for others).
Below, we’ll explain exactly what greedy algorithms are, how they work, and when and how to implement them. We’ve also included a handy cheat sheet so you can check their space-time complexity at a glance.
Here’s an outline of what we’ll cover:
Let's get into it.
1. Greedy algorithms basics
In order to crack questions about greedy algorithms, you’ll need to have a strong understanding of the technique, how it works, and when to use it.
1.1 What are greedy algorithms?
A greedy algorithm is an algorithmic paradigm that finds the optimal solution to a problem by breaking the problem down into smaller (local) parts and finding the best solution for each of these parts.
Greedy algorithms are best applied to problems with the following two properties:
- The optimal solution to the problem can build on the optimal solutions to smaller pieces of the problem. This property is known as having an optimal substructure.
- The global optimum can be reached by choosing the local optimum of each sub-problem without the need to consider future implications. This is called the greedy choice property.
An activity selection problem can be used to demonstrate these properties. Suppose we have the following options of activities to complete for the day and the optimal solution is to complete as many activities as possible:
The problem can be broken into smaller parts, for example, choosing the best activity to complete first and the remaining time become sub-problems. The best activity to complete first is the one that will finish first. Activity A finishes first at 9:30, so it is the local optimum, and the time range of 9:30 to 14:00 is the new sub-problem to solve.
The B activity falls outside this sub-problem window, so it is removed from the possible solutions set. This makes C the new local optimal, and the global problem is solved with A and C. A total of two activities are completed.
Selecting the correct heuristic for the greedy algorithm to base its choice on is critical to the functioning of the greedy algorithm. If we break the greedy choice property by using a heuristic for the local best that does not lead to a global best, then we will not find an optimal solution. For example, in the diagram below, if an activity is chosen based on having the shortest timespan, only one activity will be completed, although it is possible to complete two:
Because the highlighted activity requires the least time, it will be chosen. But this will remove the other two activities from the solution set. Instead, the "finish-first heuristic" of our first example would have selected the two unhighlighted activities.
Therefore, greedy algorithms can lead to non-optimal solutions. The only way to prove optimal solutions is to prove the greedy choice property, which is not always possible.
1.1.1 Use cases for greedy algorithms
Some of the problems greedy algorithms can be applied to are:
- Activity selection problem
- Shortest / longest path problems
- Decision tree learning
- Huffman encoding
- Minimum spanning tree problems
- Fractional knapsack problem
- Job scheduling in operating systems
- Traveling salesman problem
- K-centers problem
- Graph coloring
1.1.2 Example implementation
Implementations of greedy algorithms are similar to depth-first searching (DFS), except that only the single best solution is expanded in a greedy algorithm, whereas DFS will expand all the children of a node.
For the activities problem, this gives the following implementation:
The first part defines an activity class and its less than (<) operator so that it can be used in a sort later. The optimal_activities function makes a copy of the original list so as not to override it, and then sorts it to place the best activities first. The loop takes the best available activity that does not start before the last chosen activity’s end time. Else it becomes part of the solution, since it is the local optimal. Since this solution only has a single loop, it has a time complexity of O(n) plus the complexity for sorting.
1.1.3 How greedy algorithms compare to other algorithms
The most basic alternative to a greedy algorithm is a brute-force algorithm, which looks at all the possible solutions to a problem. But using a brute-force approach in our activity selection example will have a time complexity of O(n!).
Greedy algorithms are different from divide-and-conquer algorithms, but are related in terms of their need to solve a smaller sub-problem. Divide-and-conquer algorithms require an optimal substructure (but not the greedy choice property). Greedy algorithms can be simpler to implement than divide-and-conquer, as you only need to make a single choice at any given step.
Greedy algorithms tend to use less space than dynamic programming solutions, but dynamic programming can return all the optimal solutions, not just one. Dynamic programming properties (optimal substructures) are easier to prove than the greedy choice property.
The greedy choice property means that a greedy algorithm never looks back at its past solutions. In contrast, a backtracking algorithm will reconsider past solutions whenever it hits a dead end.
Greedy algorithms are a good choice when they are guaranteed to find the optimal solution to a problem, or at least a good enough solution. Their main benefit over other algorithms is speed and simplicity.
2. Greedy algorithms cheat sheet
You can download the cheat sheet here.
2.1 Related algorithms and techniques
2.2 Cheat sheet explained
The cheat sheet above is a summary of information you might need to know for an interview, but it’s usually not enough to simply memorize it. Instead, aim to understand each result so that you can give the answer in context.
The cheat sheet is broken into time complexity and space complexity (the processing time and memory requirements for greedy algorithms). Greedy algorithms are related to several data structures, which are also included in the cheat sheet.
2.2.1 Time complexity
In the implementation of our example activity selection problem, a single loop and a sort gave our greedy algorithm a time complexity of O(n log n). In general, greedy algorithms need to evaluate the local optimum and will sort the solution space, for a time complexity of O(n log n) outside a loop. Alternatively, a greedy algorithm can use a priority queue like a heap, with a time complexity of O(log n), in a loop, with a time complexity of O(n log n).
The greedy implementation of Huffman coding is an example of using a heap to store the weight of each encoding symbol to determine its final bit string. Each symbol needs to be iterated over, log(n), and each iteration will insert or update the heap, O(log n). This gives a total time complexity of O(n log n).
Prim’s algorithm is another example of a greedy algorithm using a heap to find a minimum spanning tree (MST) for a weighted undirected graph. Prim’s algorithm does this by keeping a heap of vertices to add to the MST next, O(log |V|) to add, update or pop from the heap, while continuously looping over the edges of the smallest vertex in the heap, O(|E|). The vertices connected to these edges are only added to the heap if they are not already in MST, O(|V|), for a total time complexity of O((|V| +|E|) log |V|).
2.2.2 Space complexity
Our example activity selection problem implementation only has a copy of the original activities list, O(n), and a solutions list that will at worst contain all the activities, O(n). This gives it a total space complexity of O(n). In general, using an extra sorted list or a heap will only take n extra space for a space complexity of O(n).
The Huffman coding heap will store the weights of each distinct symbol (S). If every symbol is unique, in the worst case, the heap will have a total of |S| leaves, and therefore the heap can have 2|S|-1 nodes. This gives a space complexity of O(|S|).
Prim’s algorithm will contain all the vertices in the MST when the heap is exhausted – |V|. At worst, all the vertices may be added to the heap on the first expansion, to give the heap a max size of |V|-1. This gives a total space complexity of O(|V|).
2.2.3 Related algorithms and data structures: sorting algorithms and heaps
Greedy algorithms need a way to find the local optimum, which is easily done when the solution space is sorted or in a priority queue. On average, sorting algorithms perform at O(n log n) time. Choosing the wrong sorting algorithm for the scenario, like Quicksort, can degrade the performance to O(n^2).
Heaps are a good choice for priority queues, and the operations most likely to be used have a time complexity of O(log n). However, interaction with the heap is likely to happen within a loop over all the elements, to give the same performance of sorting at O(n log n).
3. How to prepare for a coding interview
3.1 Refresh your knowledge
Before you start practicing interviews,you’ll want to make sure you have a strong understanding of not only greedy algorithms but also the rest of the relevant algorithms and data structures. Check out our guides for detailed explanations and useful cheat sheets.
Algorithms explained:
- Depth-first search
- Breadth-first search
- Binary search
- Sorting
- Dynamic programming
- Backtracking
- Divide and conquer
Data structures explained:
3.2 Practice on your own
Once you’re confident in all of these, you’ll want to start working through lots of coding problems.
Check out the articles in our algorithms questions series for practice on the most important algorithms:
- Depth-first search interview questions
- Breadth-first search interview questions
- Binary search interview questions
- Sorting interview questions
- Dynamic programming interview questions
- Greedy algorithm interview questions
- Backtracking interview questions
- Divide and conquer interview questions
We also recommend working through our list of 73 data structure interview questions.
One of the main challenges of coding interviews is that you have to communicate what you are doing as you are doing it. Talking through your solution out loud is therefore very helpful. This may sound strange to do on your own, but it will significantly improve the way you communicate your answers during an interview.
It's also a good idea to work on a piece of paper or google doc, given that in the real interview you'll have to code on a whiteboard or virtual equivalent.
3.3 Practice with others
Of course, if you can practice with someone else playing the role of the interviewer, that's really helpful. But sooner or later you’re probably going to want some expert interventions and feedback to really improve your coding interview skills.
That’s why we recommend practicing with ex-interviewers from top tech companies. If you know a software engineer who has experience running interviews at a big tech company, then that's fantastic. But for most of us, it's tough to find the right connections to make this happen. And it might also be difficult to practice multiple hours with that person unless you know them really well.
Here's the good news. We've already made the connections for you. We’ve created a coaching service where you can practice 1-on-1 with ex-interviewers from leading tech companies. Learn more and start scheduling sessions today.
Any questions about greedy algorithms?
If you have any questions about greedy algorithms or coding interviews in general, don't hesitate to ask them in the comments below. All questions are good questions, so go ahead!