Big O Notation

Fish & Chip

Arch-Supremacy Member
Joined
Nov 1, 2010
Messages
12,527
Reaction score
1
Hi guys,

I just got introduced to Big O notation.

From my understanding of the following block of code

Code:
int m;
for (int i = 0; i < n; i++) {
m = n;
while (m > 1) {
m = m / 2;
} // end of while
} // end of for

This produces O(N LogN). (Pls correct me if im wrong)

However, I am tasked to find the Big O Notation for the following block of code, which is very similar to the previous block.

Code:
int k = n; 
while (k > 0) {
for (int j = 0; j < n; j++) {
System.out.println(“Inside the inner loop”);
}
k = k / 2;
}

This time, the halving of data is done outside the inner forloop but within the outer whileloop.

Based on what I roughly know, I deduced it to be O(N^2 * LogN) = O (N^2 LogN)

Appreciate if anyone can guide or enlighten me on what is the Big O Notation for the 2nd block. Thanks in advance! :D:o
 

KnightNiwrem

Senior Member
Joined
Jun 1, 2014
Messages
1,057
Reaction score
0
Hi guys,

I just got introduced to Big O notation.

From my understanding of the following block of code

Code:
int m;
for (int i = 0; i < n; i++) {
m = n;
while (m > 1) {
m = m / 2;
} // end of while
} // end of for

This produces O(N LogN). (Pls correct me if im wrong)

However, I am tasked to find the Big O Notation for the following block of code, which is very similar to the previous block.

Code:
int k = n; 
while (k > 0) {
for (int j = 0; j < n; j++) {
System.out.println(“Inside the inner loop”);
}
k = k / 2;
}

This time, the halving of data is done outside the inner forloop but within the outer whileloop.

Based on what I roughly know, I deduced it to be O(N^2 * LogN) = O (N^2 LogN)

Appreciate if anyone can guide or enlighten me on what is the Big O Notation for the 2nd block. Thanks in advance! :D:o

For the first block, you would get "unbound variable n". :o

For the second block, explain your answer and we can see where you got it wrong.
 

Fish & Chip

Arch-Supremacy Member
Joined
Nov 1, 2010
Messages
12,527
Reaction score
1
For the first block, you would get "unbound variable n". :o

For the second block, explain your answer and we can see where you got it wrong.

Hm. I just got introduced to it so we havent touched anything that is related to "unbound" or what so ever. We are just beginning the basic which is to identify what is the big o notation of a given block of code. :s13:

What I understood so far is that there are 2 types of complexity, addition and multiplication. It has been clarified that the 2nd block has a complexity of multiplication.

Also what I understood is to drop the lower order term and constant.

To be honest, the Big O Notation that I came up with for the 2nd block is based on observation. This is how I derived my answer for the 2nd block.

AVQo4Xi.png


The answer for 2nd block is based on the observation of the following from my lecture notes.

hxeRsgo.jpg
 

KnightNiwrem

Senior Member
Joined
Jun 1, 2014
Messages
1,057
Reaction score
0
Hm. I just got introduced to it so we havent touched anything that is related to "unbound" or what so ever. We are just beginning the basic which is to identify what is the big o notation of a given block of code. :s13:

What I understood so far is that there are 2 types of complexity, addition and multiplication. It has been clarified that the 2nd block has a complexity of multiplication.

Also what I understood is to drop the lower order term and constant.

To be honest, the Big O Notation that I came up with for the 2nd block is based on observation. This is how I derived my answer for the 2nd block.

The answer for 2nd block is based on the observation of the following from my lecture notes.

Oh no, no, no. The "unbound variable n" was just a joke. It's because you didn't declare any variable called 'n', when you used it in your for loop. It'll return an error, saying that it couldn't find the value for that variable called 'n'. :D

You're right to say that the 2nd block has a "complexity of multiplication", if you want to call it that. I would normally just describe in a more.. err... "layman" way. Lol.

Yes. Do drop lower order terms and constants.

-------------------

Now, I think it is kind of clear where your issue lies.

For starters, you cannot assign

Code:
while(k > 0) { ...

with any big O notation without understanding how many times the loop is carried out. A while/for loop doesn't necessarily have O(n) run time complexity. Instead, the run time complexity of any loop is determined by how many times the loop has to run.

Here's an easy example:

Code:
int k = 3;
while (k > 0) {
  System.out.println("How many lines have I printed?");
}

If you take a closer look, you would easily recognise this to be an infinite loop. The program never terminates. It has a "run time complexity" of O(Infinity). The value of k does not affect the run time complexity of this algorithm in any particular way. This shows that you cannot immediately say a while/for loop has a run time complexity of O(n) carelessly.

Next, we have the most commonly seen example:

Code:
int k = 0;
int n = 5;

while (k < n) {
  System.out.println("This is line number: " + k);
  k = k + 1;
}

This is completely equivalent (for most intent and purposes) to:

Code:
int n = 5;

for (int k = 0; k < n; k++) {
  System.out.println("This is line number: " + k);
}

In both cases, the run time complexity is determined by the value of the variable 'n'. It has a linear relationship. If n is 5, the loop runs 5 times. If n is 20, the loop runs 20 times. We say that it has a run time complexity of O(n).

Finally, here's the part where you are somewhat confused about. We have:

Code:
int n = 5;

while (n > 0) {
  System.out.println("I am a line!");
  n = n / 2;
}

We can always rewrite this as:

Code:
for (int n = 5; n > 0; n = n/2) {
  System.out.println("I am a line!");
}

For both the while and for loop, in this case, their run time complexity is O(logn).

The characteristics of a O(logn) run time complexity, is one where you need to square the value of n, to double the number of operations done. This makes use of the mathematical derivation:

Code:
x = log(n)
2x = 2log(n) = log(n^2)

Here's a little demonstration to prove it. With the while loop:

Code:
while (k > 0) {
  System.out.println("OP");
  k = k / 2;
}

We run this with k = 7. We get:

Code:
OP         // k = 7/2; here
OP         // k = 3/2; here
OP         // k = 1/2; here

We square the value of k, such that k = 49. We get:

Code:
OP         // k = 49/2; here
OP         // k = 24/2; here
OP         // k = 12/2; here
OP         // k = 6/2; here
OP         // k = 3/2; here
OP         // k = 1/2; here

Hence, this demonstrates the relationship between the value of n, and the number of loop operations executed, for a O(logn) run time complexity.
 

Fish & Chip

Arch-Supremacy Member
Joined
Nov 1, 2010
Messages
12,527
Reaction score
1
Hi, thank you so much for taking your time to clarify my doubts. It is much clearer than that what i heard in school.

So as such, referring back to block 2, i can restructure the code as, where n assumes to be 5,

Code:
For (int k = 5; k >0; k = k/2) { 
   For (int j = 0; j < 5; j++) {
     //print
   }
}

So based on this, the big o notation derived is O (Log(N^2)) where the outer loop has a O (LogN) complexity where the inner loop has a O (n) complexity due to linear relationship. Am i right?

Thanks again!! :)
 

KnightNiwrem

Senior Member
Joined
Jun 1, 2014
Messages
1,057
Reaction score
0
Hi, thank you so much for taking your time to clarify my doubts. It is much clearer than that what i heard in school.

So as such, referring back to block 2, i can restructure the code as, where n assumes to be 5,

Code:
For (int k = 5; k >0; k = k/2) { 
   For (int j = 0; j < 5; j++) {
     //print
   }
}

So based on this, the big o notation derived is O (Log(N^2)) where the outer loop has a O (LogN) complexity where the inner loop has a O (n) complexity due to linear relationship. Am i right?

Thanks again!! :)

Hi. The outer loop does have O(logn) and the inner loop does have O(n). However, when you multiply them, you should get O(nlogn).

Also note that O(logn^2) doesn't exist because (log(n^2)) === (2log(n)). In big O, we can always take out constant factors, so that becomes just O(logn).
 

Fish & Chip

Arch-Supremacy Member
Joined
Nov 1, 2010
Messages
12,527
Reaction score
1
Hi! Thanks for correcting me. Now i get it. But 1 thing i still dont, if o (LogN) multiply o (n) is o (nLogN), what is the result of addition instead? Is it o (n) since it dominates o (LogN)?
 
Last edited:

KnightNiwrem

Senior Member
Joined
Jun 1, 2014
Messages
1,057
Reaction score
0
Hi! Thanks for correcting me. Now i get it. But 1 thing i still dont, if o (LogN) multiply o (n) is o (nLogN), what is the result of addition instead? Is it o (n) since it dominates o (LogN)?

Yes. O(n + logn) === O(n).

Drop lower order terms.
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ Forums. Forum members and moderators are responsible for their own posts. Please refer to our Community Guidelines and Standards and Terms and Conditions for more information.
Top