WEB HOSTING COMPANY TO RECOMMEND!

stupidbodo

Junior Member
Joined
Mar 26, 2010
Messages
89
Reaction score
0
Have you look at Elastic Beanstalk ? However if it is about ROR then it's not found natively in beanstalk, if really it's required, nothing is stopping you from having one in AWS. Look at Ruby Stack Cloud Hosting, Installers and Virtual Machines.

However, curiously, or in fact, it's a pity wondering when it becomes installing linux and applications in it becomes a system engineer job or the infra team job, and one actually need an expert for such stuffs ? So a software developer shouldn't be proficient in such tasks ? No wonder the IT standard in this part of the world is going downhill....


I like the part on "No wonder the IT standard in this part of the world is going downhill...." :s13:
 

expertleong

Junior Member
Joined
Apr 21, 2006
Messages
25
Reaction score
0
It depends on needs of individuals/company. I am not discouraging all developers to avoid basic sysadmin skills. The suggestion was made based on understanding of NSforSG's situation. He has not made up his mind on what programming language to learn, I concluded that he is still/will be learning programming language. He is not sure about various hosting options and unfamiliar with terms like "VPS". It sounds like he is new to sysadmin. I suggest he concentrate on learn programming and developing the web app first at this stage. In future when he needs more sophisticated configuration or need to scale/optimize his app, then he can start looking into solutions like AWS/Openshift.

It is a plus, if a programmer has sysadmin knowledge. But I do not think this is very essential skill to have, if the company already has a dedicated sysadmin. Sometime there is a need for specialization in certain field, there are also time where there is a need for a person to be jack of all trades.

"No wonder the IT standard in this part of the world is going downhill...." I do have similar observation on Singapore's software developers. I think this is mainly due to lack of interest in software development. Programmers don't get much respect here. And most of the Computer Science graduates are not really passionate about programming in the first place, programming is just a job to many programmers here. I can understand why Software Development field is going downhill. This article pretty sums it up quite well: Rise of the Software Craftsmen - SGE : SGE
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
It depends on needs of individuals/company. I am not discouraging all developers to avoid basic sysadmin skills. The suggestion was made based on understanding of NSforSG's situation. He has not made up his mind on what programming language to learn, I concluded that he is still/will be learning programming language. He is not sure about various hosting options and unfamiliar with terms like "VPS". It sounds like he is new to sysadmin. I suggest he concentrate on learn programming and developing the web app first at this stage. In future when he needs more sophisticated configuration or need to scale/optimize his app, then he can start looking into solutions like AWS/Openshift.

The approach while feasible, in my opinion is not ideal. As much as today we are talking about using the cloud, it doesn't really mean things must start halfway from the cloud. The knowledge to develop properly in the cloud boils down to how deep one understands the fundamentals that trace all the way back into the soil.

As you have mentioned, TS haven't decided on which programming language to approach, which certainly means it's a start. How far can development goes without understanding the platform that one is riding on ? It's like stepping on someone's shoulder and keep climbing, eventually when the bottom layer starts to alter, the top just crumble with nothing to hold on to.

That's why my perception to cloud development is, because one has the fundamentals to properly develop from ground up that makes riding on the cloud a swift. Without mentioning which platform one starts development on, it goes without saying one need to understand how a system runs, and where the code executes, how it's run.

If it's a proper advice to a new comer, it should be always start low and move up the chain. If one can't command the underlying platform directly below it, one have not earn the rights to move up the chain. If one can't even understand how to setup the platform to execute the codes, then one better put it as first priority. If not, there is nothing about scaling and optimisation that matters. It's the foundation that matters.

It is a plus, if a programmer has sysadmin knowledge. But I do not think this is very essential skill to have, if the company already has a dedicated sysadmin. Sometime there is a need for specialization in certain field, there are also time where there is a need for a person to be jack of all trades.

I really don't think it's a "plus" that a programmer has the so called "sysadmin" knowledge. How is operating an operating system just a sysadmin knowledge ? I'm puzzled. Everyone need a system to work on, so suddenly knowing how to operate in your own laptop or desktop, which could be either Windows, Mac, Linux becomes a system administrator role ? If it is indeed a system administration role, then it better be one of the criteria to put into the resume of a developer because I don't understand how can a developer earn the rights to be known as one without the skill set of managing the very layer that make things tickle.

I do understand where you are coming from when proposing one doesn't need to be jack of all trades, I believe somewhat in that too should one doesn't want to earn the title of being versatile. However I probably will place those under the differentiating between a BI expert, DB expert, security expert etc as been different verticals. I hardly call knowing how to manage the operating system an expert role. It shouldn't even be a vertical by itself.
 
Last edited:

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
I think overall it depends on what you are trying to achieve. I see many misconception that you only need to know how to write features to build a web app, unfortunately, that is not the case.

Writing a web app is easy, but writing a web app that can scale efficiently is really hard.

Suppose a new developer comes in and wants to build a new groupon clone. He picked up lessons from codecademy and begins writing his app. Few months later he finished his app and deploys it.

What is misleading is that many people thinks that once they finished writing the app they are done. Once your site turns operational and hot that's where the trouble comes in. Queries that normally takes sub milli seconds to work on your pc is now running minutes on your site.

How you write your code(whether is it spaghetti or course meal), how you deploy it(how many clusters, how many database servers, how your queries get passed) all these matters alot if you want to go big.

So far I agree with what you have conveyed. Indeed there are much deeper consideration going big. :)

It is best you think of deployment in long term. I give you a very good scenario if you are going hosted and your site gets so awesome that you want to do some data analysis from your users, you decided to deploy a data warehousing system. You begin looking at options and found this bloody cheap and awesome AWS Redshift that costs only $1000/yr/tb (Amazon Redshift) . But when you read the faq, you realized that
in order to import your data you need to either use S3 to do it free or sent it via data pipeline. If you intend to use data pipeline this deal doesn't seem so great anymore as you will need to incur additional expense.

How is it so, just because you need to incur additional expense if you choose to use data pipeline and it becomes unattractive anymore ? Doesn't seems to be what I understood about Redshift. I think if you today compare Redshift alone to any traditional large sized BI solution, it is already a steal. The fact that you can shutdown the instances and then save your cost, how would a traditional data centre solution be able to save on the invested cost ?

When you say using data pipeline, may I understand where is the source of your data transfer ? Are you referring to the on premise data ? If so, would AWS Import be feasible for your scenario ?

Even so, you might want to be more thorough in your TCO. Having your own BI solution besides the software licensing and the infrastructure investment, you also need to invest in system engineers, solution architect to keep the system in tip-top condition. It's not like Redshift doesn't need a solution architect, but the need for system engineers managing a on-premises solution may not be required after all. Even if they are indispensable, the cost in getting personnels with niche skill sets might not be as important using Redshift versus traditional approach.

Every vendor has their pros and cons, I don't support any because I use it based on what I am trying to do. Heroku is good because it is easy to deploy and once you scale out you don't have to waste so much time thinking about how to configure multiple servers. Some people might say its expensive but depending on your case, you might saved it on labor costs.


I am a wheel dealer, what I do is I get a dirt cheap hourly priced hosted or cloud service like AWS for development (why aren't you using that? heck you can do all sorts of funny things).

BUT when it comes to real deployment, I make sure I planned carefully so that I don't incur so much technical debt (Technical debt - Wikipedia, the free encyclopedia) .

Unfortunately if you want to build a successful app, you have to at least understand what is the best way to deploy your site. Okay, back to writing code.

It's definitely good to know how to position your solution not only technically, but also understanding the business concepts involved. After all technology is a business enablement tool. There is no stopping from one to use several cloud solutions in the whole solution ecosystems, as business and solution redundancies. Just calculate the cost and efforts require to sustain such an approach. If it make sense, why not. There are always times to be prudent.
 
Last edited:

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
"No wonder the IT standard in this part of the world is going downhill...." I do have similar observation on Singapore's software developers. I think this is mainly due to lack of interest in software development. Programmers don't get much respect here. And most of the Computer Science graduates are not really passionate about programming in the first place, programming is just a job to many programmers here. I can understand why Software Development field is going downhill. This article pretty sums it up quite well: Rise of the Software Craftsmen - SGE : SGE

I just thought I want to put this as a separate post since it is a relatively different nature from what TS wants.

Now about this software development endeavour, what I feel is people nowadays is greedy. The greed is want more than what they are worth. Reading other forums in HWZ at times, I read about some whom are either still studying or just graduated asking how much they are worth in the industry and how much they should ask for in new company. Then they start comparing whom someone they knew got a higher pay than what is offered to them.

My first thought is "what the heck ? Just a piece of white paper..." If you ask me, 20 years ago when poly students and degree holders are cream of the crop, perhaps such conversations make sense. Today where annually universities are churning out tens of thousands of fresh graduates, perhaps it's time to set a more realistic expectation.

Holding on to the fresh piece of certificate means nothing. It just means you got a entry ticket. To make it in the industry or break it depends on first and foremost ATTITUDE, second COMMITMENT, and third CAPABILITIES. Without the right attitude, you drown before you start. Without commitment, you drown before you are halfway in the swim. Without capabilities, you can't swim far or even reach the land.

But so far, when I read about those comments asking how much they are worth, immediately they failed in ATTITUDE. Fortunately or perhaps unfortunately, not everyone is cut out to be a technologist. If they have no passion, then we don't need them.

Respect is something to be earned. If you want to be respected, then learn to appreciate in what you are doing and respect what you are doing. It's not just programmers or not, it's in every industries.

I think we need to be realistic too. Programmers are high level construction workers. While building construction workers work their asses under the sun, programmers work their asses around the clock. They are different kind of hardships. Face it, the only roles in this industry that is going to earn some prominent respect are managerial roles. If you want to be a big time in this industry, know how to command. A good commander have a lot more to offer and larger influence in the company. When I say managerial roles, it doesn't need to be a manager or a director. It can be a good technical project manager/director, a good solution architect or consultant, a good tech lead, or maybe a CTO or CIO. I know not all candidates of these roles live up to expectation, but if your aim is to excel and have bigger influence over things around you, these are the designations that will give you some level of satisfaction. So it seems in this part of the world, that's how it operates.

Something we really need to learn from the westerns. When the rest of the world compete with them, their attitude is we are better ? (Whether this is true or not is not the point). The point is the ATTITUDE! "We are worth more and we are better because we can deliver better!" That's that kind of impression you get from them.

But here is different. When our rice bowl is shaken... Why are you fighting with me ? Why are you underquoting my worth ? I hate you because you use a more affordable price to fight with me.

See the difference in attitude ? It's not how foreigners manage to get into our industry. Instead of having fighting spirit to hold your head high, to make your stand in the homeland, to have attempts to improve your own worth, it ends up questioning why others can do the same job as you are at halve your price. Instead of telling your boss that you can deliver more, you end up asking your boss why he choose the foreigners over you. That's where it all fails seriously.

When the strong comes, the correct attitude is we must be stronger. Fight back with skill sets, fight back with knowledge. That's how things will improve. Hence with respect to the article found at Rise of the Software Craftsmen - SGE : SGE, my opinion towards foreigners coming into our land and taking over our place, sometimes we need to ask ourselves this question.

What have we done to deserve more ?
 
Last edited:

NSforSG

High Supremacy Member
Joined
Nov 17, 2010
Messages
34,250
Reaction score
2
err... sorry guys. didn't mean to create a "heated argument" over my situation. (i will rather see it as voicing out individuals' opinions and taking it as learning points)

expertleong may be an expert in IT field, likewise for any other users who commented here. I won't dare to admit i'm a good programmer, let alone an expert. Hence I humbly want to learn from you guys.

What happened was my project group created a website for price comparison in Singapore (but i did almost all the coding). it's meant to be a project that needs to be submitted, and i love doing it, though it's just 10% of the module... :( the website can be viewed here: Open Market - It's time to do justice to your pockets! (there may be a lot of bugs and security flaws, i know, but it's just 10% and the module is not about creating a website...)

moving on from here, i want to create another website. this time round i may want to use ruby on rails to manage the server side part. hence, i'm looking for a web hosting that has the capabilities to host RoR.

A little more about myself: i got in touch with programming since primary school, when i join the IT club. For me, it's like HTML, javascript, then css. then moving on to trying PHP, MySQL, Perl, CGI, ASP, XML, XHTML, Java, C, C++, Visual Basic, Flash, etc. I don't really know what is the real meaning of learning a programming language, because if you ask me to write a code in those languages mentioned, i can't do it without a manual, but if you were to give me a day, i can implement them with the help of the manual.

is learning a programming language really mean to memorise the syntax, word by word? i don't know. haha.

in the end, i didn't take programming course in uni. kind of regret not doing so, so i try to take some computing modules to 'compensate'. haha.

i learn most of them through borrowing books from the public libraries, hence there may be many technical terms i don't know. However, I hope fellow users can teach me, so that i can learn along the way. :)
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
err... sorry guys. didn't mean to create a "heated argument" over my situation. (i will rather see it as voicing out individuals' opinions and taking it as learning points)

expertleong may be an expert in IT field, likewise for any other users who commented here. I won't dare to admit i'm a good programmer, let alone an expert. Hence I humbly want to learn from you guys.

What happened was my project group created a website for price comparison in Singapore (but i did almost all the coding). it's meant to be a project that needs to be submitted, and i love doing it, though it's just 10% of the module... :( the website can be viewed here: Open Market - It's time to do justice to your pockets! (there may be a lot of bugs and security flaws, i know, but it's just 10% and the module is not about creating a website...)

moving on from here, i want to create another website. this time round i may want to use ruby on rails to manage the server side part. hence, i'm looking for a web hosting that has the capabilities to host RoR.

A little more about myself: i got in touch with programming since primary school, when i join the IT club. For me, it's like HTML, javascript, then css. then moving on to trying PHP, MySQL, Perl, CGI, ASP, XML, XHTML, Java, C, C++, Visual Basic, Flash, etc. I don't really know what is the real meaning of learning a programming language, because if you ask me to write a code in those languages mentioned, i can't do it without a manual, but if you were to give me a day, i can implement them with the help of the manual.

is learning a programming language really mean to memorise the syntax, word by word? i don't know. haha.

in the end, i didn't take programming course in uni. kind of regret not doing so, so i try to take some computing modules to 'compensate'. haha.

i learn most of them through borrowing books from the public libraries, hence there may be many technical terms i don't know. However, I hope fellow users can teach me, so that i can learn along the way. :)

Debating it may be, "heated argument" I'm not in the mood, even though these few nights are hot in Singapore.

Lets expand the scope a bit, don't call it learning a programming language. Call it learning how to program or how to develop software. In this way, we can cover more grounds and you can achieve what you eventually set out for.

There are tons of programming languages out in the industry, you can name them from A-Z easily. However only a couple of programming languages get a good foothold in the industry either support from the public communities or enterprises.

What you really need is skill set to develop properly in an environment. It's commendable you have interest in development and making good attitude to learn. The big objective here is knowing your tools and knowing your concepts. This is the flexible portion of development that will bring you almost everywhere. Programming languages are just tools to express your intention. You know english, and you probably also knew mandarin. Either way, they get messages across. Likewise all programming languages do. You don't need to read up a manual to type this article is because you memorize by frequently using them. It is exactly the same for programming languages. I do programming for more than a decade and still I rely on references from time to time, it's normal. With so much API and languages, how can one remember all. What's important is be flexible and be as thorough as you can be.

My perspective is regardless which platform you have chosen to operate on, learn how to make your ways in it. You don't know where and what your playground is, you can't play well. As simple as that.

Nowadays I hardly hear people visiting public libraries, so it's good you do. Reading materials of all sort from database, programming, to even understanding about office software give you good breadth about technologies around you. They will come in handy at times. Read up more in googling. Come across some weird keywords about technologies, google it and get yourself away from the normal stuffs you read, you will end up better in all ways, not just software development.
 

NSforSG

High Supremacy Member
Joined
Nov 17, 2010
Messages
34,250
Reaction score
2
Debating it may be, "heated argument" I'm not in the mood, even though these few nights are hot in Singapore.

Lets expand the scope a bit, don't call it learning a programming language. Call it learning how to program or how to develop software. In this way, we can cover more grounds and you can achieve what you eventually set out for.

There are tons of programming languages out in the industry, you can name them from A-Z easily. However only a couple of programming languages get a good foothold in the industry either support from the public communities or enterprises.

What you really need is skill set to develop properly in an environment. It's commendable you have interest in development and making good attitude to learn. The big objective here is knowing your tools and knowing your concepts. This is the flexible portion of development that will bring you almost everywhere. Programming languages are just tools to express your intention. You know english, and you probably also knew mandarin. Either way, they get messages across. Likewise all programming languages do. You don't need to read up a manual to type this article is because you memorize by frequently using them. It is exactly the same for programming languages. I do programming for more than a decade and still I rely on references from time to time, it's normal. With so much API and languages, how can one remember all. What's important is be flexible and be as thorough as you can be.

My perspective is regardless which platform you have chosen to operate on, learn how to make your ways in it. You don't know where and what your playground is, you can't play well. As simple as that.

Nowadays I hardly hear people visiting public libraries, so it's good you do. Reading materials of all sort from database, programming, to even understanding about office software give you good breadth about technologies around you. They will come in handy at times. Read up more in googling. Come across some weird keywords about technologies, google it and get yourself away from the normal stuffs you read, you will end up better in all ways, not just software development.

a lot of good points, and i appreciate them. Suddenly I feel like asking and learning more, but it's totally off from this thread topic! Maybe it's better for me to start a new thread? :)
 

expertleong

Junior Member
Joined
Apr 21, 2006
Messages
25
Reaction score
0
If it's a proper advice to a new comer, it should be always start low and move up the chain. If one can't command the underlying platform directly below it, one have not earn the rights to move up the chain.

When giving suggestion, one should be making decision based on the eventual goal, not based on personal ideology. Let's look at NSforSG's intention. He is a newbie to web development, and his goal is a publish a web app. Are you suggesting he go study server technologies, and infrastructure instead of picking up a language for web programming? Supposed I want to publish a simple Android, am I supposed to learn ARM programming and how to patch Linux kernel first before looking at Android SDK?

When you said "start low" are you talking about low-level stuffs like hardware and OS? By "move up the chain", are you referring to stuffs like application programming? At the end of the day, from user's perspective, they don't care how a web app is built, neither are them interested in how it is hosted. If I have a great app idea that solves my user's problem while giving them a great UX, I can build a awesome app even without learning how machines works.

There is a purpose why IaaS and PaaS exists. They exist to serve different purpose. If you have a limited resource/knowledge and your intention is to publish a web app, of course it make sense to go with PaaS like Heroku. Heroku was design to help people who want to concentrate on building web apps, and help them manage concerns like scaling and infrastructure.

When a newbie's goal is to learn a programming language and publish a web app, why would it be ideal to "always start low and move up the chain"? At this stage, his goal is not to build data center or become a infrastructure engineer, he just want to learn basic web programming and quickly publish a web app.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
When giving suggestion, one should be making decision based on the eventual goal, not based on personal ideology. Let's look at NSforSG's intention. He is a newbie to web development, and his goal is a publish a web app. Are you suggesting he go study server technologies, and infrastructure instead of picking up a language for web programming? Supposed I want to publish a simple Android, am I supposed to learn ARM programming and how to patch Linux kernel first before looking at Android SDK?

Lets be realistic here, knowing how to install software on a linux system doesn't make you a system administrator. So where my foundation line is drawn, it is not something out of the blue. It has always been done for more than a decade.

Back in those days, these developers are also known as Web developers. So suddenly the skill set of a web developer just got narrow down a lot because PaaS is born ? I don't think so. Pick up a few web development books and you will easily see installation of some simple tools or libraries are part and parcel of web development.

No you don't need to learn how to compile a kernel to learn web development, but you are highly encouraged or recommended to know your ways around your operating system like knowing how to edit the cron, tail the logs, manage your packages via package managers, and open or close firewalls if you want to be a web developer with a future.

That being said, there are tons of "web developers" that doesn't know all these. Throughout my career path, I have seen numerous of such half-ass developers that have little progression in their career paths. So from my standpoint, I will not recommend TS to go this way since it produce half-ass developers and also one with little future.

When you said "start low" are you talking about low-level stuffs like hardware and OS? By "move up the chain", are you referring to stuffs like application programming? At the end of the day, from user's perspective, they don't care how a web app is built, neither are them interested in how it is hosted. If I have a great app idea that solves my user's problem while giving them a great UX, I can build a awesome app even without learning how machines works.

Yes you are right, from the user perspective, they don't see all the hardships behind the back, but that doesn't mean they are not important. Lets be more realistic as a software developer. I'm teaching TS how to be a software developer, when did web development just degraded into knowing how to write a few server scripts, client scripts and stylesheets. That is pretty shallow a terminology for web developers. My appreciation for web developers are far thicker than just knowing how to script.

There is a purpose why IaaS and PaaS exists. They exist to serve different purpose. If you have a limited resource/knowledge and your intention is to publish a web app, of course it make sense to go with PaaS like Heroku. Heroku was design to help people who want to concentrate on building web apps, and help them manage concerns like scaling and infrastructure.

IaaS and PaaS exist to make development easier and more cost effective. But it has totally nothing to do with making you learn less. If that is how one perceive IaaS and PaaS, then it must be a short sighted view.

When a newbie's goal is to learn a programming language and publish a web app, why would it be ideal to "always start low and move up the chain"? At this stage, his goal is not to build data center or become a infrastructure engineer, he just want to learn basic web programming and quickly publish a web app.

More than 10 years ago, I'm also a newbie, but today, my pride as a web developer is I can command most levels of the physical and software stack from physical network cabling and configuring hardware load balancers, switches and firewalls, to the middleware like IAM/SSO, application servers, operating systems, to distributed computing and even holding discussions on web UX and UI. Web designs along with DHTML using various client side scripting techniques and libraries are also my core competencies.

To:TS

The way I look at how this debate is going, my sentiments are simple for TS. NSforSG, it's after all TS's prerogative in how TS want to learn something. Skip the bottom layer and be a web developer like what expertleong adviced, which is feasible, just not to my taste.

Take my advice and learn how to operate linux at the same time you learn web development. You will be a much stronger web developer with a better skill set, move faster in no time, and travel further. The choice is really up to you. :)
 

stupidbodo

Junior Member
Joined
Mar 26, 2010
Messages
89
Reaction score
0
So far I agree with what you have conveyed. Indeed there are much deeper consideration going big. :)



How is it so, just because you need to incur additional expense if you choose to use data pipeline and it becomes unattractive anymore ? Doesn't seems to be what I understood about Redshift. I think if you today compare Redshift alone to any traditional large sized BI solution, it is already a steal. The fact that you can shutdown the instances and then save your cost, how would a traditional data centre solution be able to save on the invested cost ?

When you say using data pipeline, may I understand where is the source of your data transfer ? Are you referring to the on premise data ? If so, would AWS Import be feasible for your scenario ?

Even so, you might want to be more thorough in your TCO. Having your own BI solution besides the software licensing and the infrastructure investment, you also need to invest in system engineers, solution architect to keep the system in tip-top condition. It's not like Redshift doesn't need a solution architect, but the need for system engineers managing a on-premises solution may not be required after all. Even if they are indispensable, the cost in getting personnels with niche skill sets might not be as important using Redshift versus traditional approach.



It's definitely good to know how to position your solution not only technically, but also understanding the business concepts involved. After all technology is a business enablement tool. There is no stopping from one to use several cloud solutions in the whole solution ecosystems, as business and solution redundancies. Just calculate the cost and efforts require to sustain such an approach. If it make sense, why not. There are always times to be prudent.


Yes I can totally understand where you are coming from. It's a good point you made about shutting down the data warehouse since some users don't need it all the time. Compared to third party BI vendors redshift is definitely disruptive and I love it when they enter the market because all third party BI tools are dropping their prices like mad (2011 at $100,000/TB to 2013 $2000-$10000/TB) to adapt.

I won't say Redshift can replace all BI tools since most of them have strength in their own ways like advanced Columnar compression, fast ad hoc querying.

Nonethelsss it's pretty impressive for Redshift to offer that kind of price with as a Columnar data store with MPP and compressions.


Perhaps my "very good scenario" isn't that good after all. Regarding the pipeline, I was actually referring to streaming data to be used for user's reporting. In this case Redshift I'm not sure if Redshift would be the best choice since the churn out period required would be around a day and latency counts. How fast Redshift returns a query is still very much unknown. Let's keep an eye for more case study on Redshift.

I agree with your last paragraph.
 

NSforSG

High Supremacy Member
Joined
Nov 17, 2010
Messages
34,250
Reaction score
2
thanks everyone for the comment. Actually I'm not the TS of this thread. you can say i'm a hijacker that hijack this thread.

learning is fun, especially learning programming. i don't mind learning. while sometimes most of us want to take the easy way out, i don't mind learning the fundamental of everything. haha.

Like learning how to program MIPS. i'm pretty sure that is really fundamental ba. (anything more fundamental, i really need to think about talking to transistors and resistors le). add $0, $1, $2 etc. quite fun. (but programming an intel chip isn't that fun. haha.)
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
Yes I can totally understand where you are coming from. It's a good point you made about shutting down the data warehouse since some users don't need it all the time. Compared to third party BI vendors redshift is definitely disruptive and I love it when they enter the market because all third party BI tools are dropping their prices like mad (2011 at $100,000/TB to 2013 $2000-$10000/TB) to adapt.

I won't say Redshift can replace all BI tools since most of them have strength in their own ways like advanced Columnar compression, fast ad hoc querying.

Nonethelsss it's pretty impressive for Redshift to offer that kind of price with as a Columnar data store with MPP and compressions.


Perhaps my "very good scenario" isn't that good after all. Regarding the pipeline, I was actually referring to streaming data to be used for user's reporting. In this case Redshift I'm not sure if Redshift would be the best choice since the churn out period required would be around a day and latency counts. How fast Redshift returns a query is still very much unknown. Let's keep an eye for more case study on Redshift.

I agree with your last paragraph.

I don't do BI much, you are probably the better candidate to know where Redshift stand in the various BI tools available in the market. My company is a consulting partner of AWS and I am currently one of the solution consultant taking care of AWS. Recently there is a partner sharing session of Redshift. The offer seems very attractive from an architectural standpoint.

If you ask me what other mature commercial BI solution features may have that Redshift doesn't offer, I probably wouldn't know much. But if you ask me if Redshift is cheaper, I have a good feeling it is.

If your company is doing BI big time, it will be a valuable experience if you contact AWS and see if it is possible to squeeze in any technical sharing sessions from them.

Regarding the performance, Redshift is marketing itself as a real contender to existing solution. You may read up from techcrunch at Hapyrus Launches Service For Amazon Redshift, An Emerging Alternative To Hadoop And Hive | TechCrunch. This is one of the performance use case shared in the partners session too.
However marketing aside, your company might want to select a couple of adventurous customer, show them the TCO numbers, solution to tackle partial of their problem and see if it can convince them for a trial. Real numbers that you can generate based on your use case is always the best matrix for consideration.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
thanks everyone for the comment. Actually I'm not the TS of this thread. you can say i'm a hijacker that hijack this thread.

learning is fun, especially learning programming. i don't mind learning. while sometimes most of us want to take the easy way out, i don't mind learning the fundamental of everything. haha.

Like learning how to program MIPS. i'm pretty sure that is really fundamental ba. (anything more fundamental, i really need to think about talking to transistors and resistors le). add $0, $1, $2 etc. quite fun. (but programming an intel chip isn't that fun. haha.)

Indeed learning is fun. Only when one attitude is right in learning is where one can gain more. Learning about how your processors works is a good insight to how machine work for you. At the same time, it also introduce you to the genius of human technologies.

If you attended a full fledge computer science course in university, you should be given a choice to take up computer architecture. This course gives you insight into how processors works at some high level. It's not low level enough to reach into electrical engineering involving clock sync and currents. These will be classified under semi conductor science and electrical engineering.

But you get to understand how complicated super high level programs function when you start breaking them down into opcodes. How memory are accessed at register level. MIPS is one of the educational assembly language due to its simplicity as a RISC processor. RISC processors are know to have a lot less variants of instruction sets performing the same task as oppose to CISC processors.

However since more than 15 years now, you find few developers understand about these fundamentals. It's really a pity. In my time, there is a project called V2OS which aim to have a OS completely written using assembly language, but it didn't make it, obviously for a good reason. Assembly languages takes too much effort to coordinate as a project of today's scale. Compilers are getting better over time to generate more efficient and compact assembly codes that obsolete most of the needs to touch at such low level, but still it's a very educational knowledge to know what you are stepping on when you are doing computing.

Back to the big question, can a web developer not knowing all these low level stuffs create solution that customers wanted and still server thousands or millions of subscribers ? The answer is yes, but will having such low level knowledge makes any difference ? The answer is YES and NO.

NO because it really doesn't require these low level knowledge to do something big. There are numerous frameworks, high level infrastructures and cloud services that shield you from the bottom level. If you know how to operate at this high level, you can actually accomplish alot upon these readily available technologies.

YES because if you aim to be the best among the rest, you need such knowledge as your basis for innovation. These few years, I see fundamentals classic computing technologies reinvent, tear down and recreated from scratch. But these are only among those with strong foundation in computing. If you can't understand why 2's complement is chosen for numeric representation, you wouldn't understand where you gain or lose if you choose something else to replace it.

If you don't understand what atomicity means at an machine instruction level, you will never understand why a simple state

Code:
i = i + 1

is not suppose to be run in a shared model environment. You probably will be warned when this is not suppose to work, but how do you understand WHY if you don't understand how such simple statements are accomplished at machine opcode level ? Yet, this fundamental is the basis of distributed computing when you need to share resources.

So you see, knowledge of fundamentals at such low level are related to distributed computing at super high level across computing nodes. When you talk about scalability that a web developer need to accomplish at high level, it somehow tests on how capable this web developer is. If you don't understand such fundamentals, you can't go far.

Why is learning the platform you are working on important ? If you don't, you are a handicapped web developer. As a web developer, one much understand a whole suite of technologies. Today web development has multiple layers of stack to cater to. It's not knowing a couple of server scripts, client scripts and DHTML makes you a web developer.

Such web developer are merely scratching the surface. Today web developers need to understand about multi service integrations, database optimisation, security enforcement, multi language integrations, a couple of middlewares, user interface design and user experience methodologies.

One thing I notice angmoh web developers are they really encompass themselves with a lot of technologies around them. They are not afraid to jump around and go in depth when necessary.

Here this part of the world, you find no lack of developers following strictly to learn on requirements. That's why this part of the world developers are going downhill. I'm in this part of the world, and I feel a pity in the culture we are living in. For whatever reasons we can find, we can blame someone or some others for the predicament we are in and will be in, but that's not solving problem.

We will always hear the "ang mohs" are better until we start stop blaming others for why we are in this state and starting changing from within. We have to start nurturing the culture of greedy for knowledge. Not afraid to venture into unknown grounds. Stop thinking if something is worth learning before you touch it.

How can one knows when a knowledge is applicable before you learn it ? I'm impressed with the kind of foresights we have at times, we can actually make the best assessment in stuffs before the knowledge is acquired. 未卜先知。

Back to the knowledge about OS. Why is it necessary ? It's because a full suite of solution is not just built using one language or one technology. No one says you should only focus on one. A simple solution I built can easily encompass 3 different programming languages.

For a recent subproject I did, I used Java as the base layer for clustering. I use perl for the actualy work done. I use a tool written in python that will be invoke by the perl codes. This subproject is suppose to interact with instagram and twitter. So would you call this a web development and am I a web developer ?

If you can't manoeuvre properly in the platform you are working on, may it be unix or windows, you can forget about getting things done. If I need to manipulate images, I can use imagemagick even without a PHP library for it. If I need to encode video or extract thumbnails from them on the fly, I use ffmpeg. If I need to access the S3, upload a file, I don't need to go through the need of slowly writting the S3 API in PHP, or Java or whatsoever. I use the s3cmd that does it for me immediately with all functionality via command options.

All these are only possible if you know your platform well. Like installation applications, setting up the linux environment, reading logs, creating directories, running services and so forth. PaaS infrastructure will deprive you of such opportunities to learn these and do more than just PaaS.

But all these been said, I have no grudge against IaaS, PaaS or SaaS. They are there for very good reasons and I love them. They make projects more affordable, they make projects development time shorter. But for self development, they make bad candidates. One must be smart to know which things are beneficial to you and which are destructive to you.

AWS is a very interesting cloud platform as oppose to the rest. It encompass all 3 aspect of IaaS, PaaS and SaaS. You can start simple, you can grow very complicated too. That's why among all the different cloud providers, I have bet on AWS to make it big. I classify my skill set as a solution architect and as often work on stuffs that I need control, yet, for some stuffs, since i already have the fundamentals, I decided I can cut cost and effort by using readily available services.

At our level, when we choose services, we need to be mindful of tie down to certain technologies. Some services may not have replacements, but you still need to know what they are and how you can mitigate these business risks. After all AWS is a business too, it can collapse, just that it's very unlikely based on their track records. If they can survive a internet balloon burst, they have what it takes to continue.

So that's all I can offer at this point. My advice is choose wisely, don't be too objective oriented when making decision. Try to look around at what MORE you can achieve.
 
Last edited:

NSforSG

High Supremacy Member
Joined
Nov 17, 2010
Messages
34,250
Reaction score
2
Indeed learning is fun. Only when one attitude is right in learning is where one can gain more. Learning about how your processors works is a good insight to how machine work for you. At the same time, it also introduce you to the genius of human technologies.

If you attended a full fledge computer science course in university, you should be given a choice to take up computer architecture. This course gives you insight into how processors works at some high level. It's not low level enough to reach into electrical engineering involving clock sync and currents. These will be classified under semi conductor science and electrical engineering.

But you get to understand how complicated super high level programs function when you start breaking them down into opcodes. How memory are accessed at register level. MIPS is one of the educational assembly language due to its simplicity as a RISC processor. RISC processors are know to have a lot less variants of instruction sets performing the same task as oppose to CISC processors.

However since more than 15 years now, you find few developers understand about these fundamentals. It's really a pity. In my time, there is a project called V2OS which aim to have a OS completely written using assembly language, but it didn't make it, obviously for a good reason. Assembly languages takes too much effort to coordinate as a project of today's scale. Compilers are getting better over time to generate more efficient and compact assembly codes that obsolete most of the needs to touch at such low level, but still it's a very educational knowledge to know what you are stepping on when you are doing computing.

Back to the big question, can a web developer not knowing all these low level stuffs create solution that customers wanted and still server thousands or millions of subscribers ? The answer is yes, but will having such low level knowledge makes any difference ? The answer is YES and NO.

NO because it really doesn't require these low level knowledge to do something big. There are numerous frameworks, high level infrastructures and cloud services that shield you from the bottom level. If you know how to operate at this high level, you can actually accomplish alot upon these readily available technologies.

YES because if you aim to be the best among the rest, you need such knowledge as your basis for innovation. These few years, I see fundamentals classic computing technologies reinvent, tear down and recreated from scratch. But these are only among those with strong foundation in computing. If you can't understand why 2's complement is chosen for numeric representation, you wouldn't understand where you gain or lose if you choose something else to replace it.

If you don't understand what atomicity means at an machine instruction level, you will never understand why a simple state

Code:
i = i + 1

is not suppose to be run in a shared model environment. You probably will be warned when this is not suppose to work, but how do you understand WHY if you don't understand how such simple statements are accomplished at machine opcode level ? Yet, this fundamental is the basis of distributed computing when you need to share resources.

So you see, knowledge of fundamentals at such low level are related to distributed computing at super high level across computing nodes. When you talk about scalability that a web developer need to accomplish at high level, it somehow tests on how capable this web developer is. If you don't understand such fundamentals, you can't go far.

Why is learning the platform you are working on important ? If you don't, you are a handicapped web developer. As a web developer, one much understand a whole suite of technologies. Today web development has multiple layers of stack to cater to. It's not knowing a couple of server scripts, client scripts and DHTML makes you a web developer.

Such web developer are merely scratching the surface. Today web developers need to understand about multi service integrations, database optimisation, security enforcement, multi language integrations, a couple of middlewares, user interface design and user experience methodologies.

One thing I notice angmoh web developers are they really encompass themselves with a lot of technologies around them. They are not afraid to jump around and go in depth when necessary.

Here this part of the world, you find no lack of developers following strictly to learn on requirements. That's why this part of the world developers are going downhill. I'm in this part of the world, and I feel a pity in the culture we are living in. For whatever reasons we can find, we can blame someone or some others for the predicament we are in and will be in, but that's not solving problem.

We will always hear the "ang mohs" are better until we start stop blaming others for why we are in this state and starting changing from within. We have to start nurturing the culture of greedy for knowledge. Not afraid to venture into unknown grounds. Stop thinking if something is worth learning before you touch it.

How can one knows when a knowledge is applicable before you learn it ? I'm impressed with the kind of foresights we have at times, we can actually make the best assessment in stuffs before the knowledge is acquired. 未卜先知。

Back to the knowledge about OS. Why is it necessary ? It's because a full suite of solution is not just built using one language or one technology. No one says you should only focus on one. A simple solution I built can easily encompass 3 different programming languages.

For a recent subproject I did, I used Java as the base layer for clustering. I use perl for the actualy work done. I use a tool written in python that will be invoke by the perl codes. This subproject is suppose to interact with instagram and twitter. So would you call this a web development and am I a web developer ?

If you can't manoeuvre properly in the platform you are working on, may it be unix or windows, you can forget about getting things done. If I need to manipulate images, I can use imagemagick even without a PHP library for it. If I need to encode video or extract thumbnails from them on the fly, I use ffmpeg. If I need to access the S3, upload a file, I don't need to go through the need of slowly writting the S3 API in PHP, or Java or whatsoever. I use the s3cmd that does it for me immediately with all functionality via command options.

All these are only possible if you know your platform well. Like installation applications, setting up the linux environment, reading logs, creating directories, running services and so forth. PaaS infrastructure will deprive you of such opportunities to learn these and do more than just PaaS.

But all these been said, I have no grudge against IaaS, PaaS or SaaS. They are there for very good reasons and I love them. They make projects more affordable, they make projects development time shorter. But for self development, they make bad candidates. One must be smart to know which things are beneficial to you and which are destructive to you.

AWS is a very interesting cloud platform as oppose to the rest. It encompass all 3 aspect of IaaS, PaaS and SaaS. You can start simple, you can grow very complicated too. That's why among all the different cloud providers, I have bet on AWS to make it big. I classify my skill set as a solution architect and as often work on stuffs that I need control, yet, for some stuffs, since i already have the fundamentals, I decided I can cut cost and effort by using readily available services.

At our level, when we choose services, we need to be mindful of tie down to certain technologies. Some services may not have replacements, but you still need to know what they are and how you can mitigate these business risks. After all AWS is a business too, it can collapse, just that it's very unlikely based on their track records. If they can survive a internet balloon burst, they have what it takes to continue.

So that's all I can offer at this point. My advice is choose wisely, don't be too objective oriented when making decision. Try to look around at what MORE you can achieve.

thanks senior, haha. wise words!

Indeed, they dare to try all sorts of new things. I'm not sure if anyone here know about this language called "Go" by Google. it's pretty new as a language, and i haven't come across any site or application using it. However, I don't mind learning it because who knows, Go may be the next popular language?

The world is changing, likewise for the computing realm. i believe we need to learn constantly in order to be on par with the changes. :)
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
thanks senior, haha. wise words!

Indeed, they dare to try all sorts of new things. I'm not sure if anyone here know about this language called "Go" by Google. it's pretty new as a language, and i haven't come across any site or application using it. However, I don't mind learning it because who knows, Go may be the next popular language?

The world is changing, likewise for the computing realm. i believe we need to learn constantly in order to be on par with the changes. :)

Yes, watched the introduction of Go in Google IO. A new kid in the block, some are pledging by it, calling it a future language. Recently a good number of languages gain some traction. To name a few that you might want to look into

1) Coffeescript
2) Scala
3) Dart
4) Go (mentioned)
5) Javascript on Node.js (not new language, but new usage)
6) F#

I bet there are a few more that I'm not aware of. There are a lot of talks Java is going to die. Each time I heard that, I feel like yawning. Not that I'm complacent. I'm quite sure Java at some point be like C where it's not the main focus where in time new languages make development easier, less boilerplates, more features, more protection and so forth.
That's how we evolve, but that been said, all these programming languages are nothing more than expressive tools at a developer disposal.

A software developer must learn to embrace more in methodologies and general skill sets to move up the chain. These skill sets can be bring across different platforms and languages. So just use whatever is available in your era and get work done. That's more important.

As for me, meanwhile my 2 objectives on programming languages are Erlang and Scala. Other part of my involvement are more on solution-ing on various platforms and looking at how to cultivate developers in my company. This industry need new blood, not just experts. We need tomorrow's experts. If we don't start cultivate now, there wouldn't be sunny days to look forward to :)
 

NSforSG

High Supremacy Member
Joined
Nov 17, 2010
Messages
34,250
Reaction score
2
Yes, watched the introduction of Go in Google IO. A new kid in the block, some are pledging by it, calling it a future language. Recently a good number of languages gain some traction. To name a few that you might want to look into

1) Coffeescript
2) Scala
3) Dart
4) Go (mentioned)
5) Javascript on Node.js (not new language, but new usage)
6) F#

I bet there are a few more that I'm not aware of. There are a lot of talks Java is going to die. Each time I heard that, I feel like yawning. Not that I'm complacent. I'm quite sure Java at some point be like C where it's not the main focus where in time new languages make development easier, less boilerplates, more features, more protection and so forth.
That's how we evolve, but that been said, all these programming languages are nothing more than expressive tools at a developer disposal.

A software developer must learn to embrace more in methodologies and general skill sets to move up the chain. These skill sets can be bring across different platforms and languages. So just use whatever is available in your era and get work done. That's more important.

As for me, meanwhile my 2 objectives on programming languages are Erlang and Scala. Other part of my involvement are more on solution-ing on various platforms and looking at how to cultivate developers in my company. This industry need new blood, not just experts. We need tomorrow's experts. If we don't start cultivate now, there wouldn't be sunny days to look forward to :)

Thanks for the recommendation. i will look at them soon! :D

I have a friend who just graduate from ntu computer science. she is now working in a bank. the system that they are using is groovy on grails. it's like Java. So perhaps i will say that Java is "morphing" into other forms? But at least i don't see signs of dying. haha.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,391
Reaction score
1,180
Thanks for the recommendation. i will look at them soon! :D

I have a friend who just graduate from ntu computer science. she is now working in a bank. the system that they are using is groovy on grails. it's like Java. So perhaps i will say that Java is "morphing" into other forms? But at least i don't see signs of dying. haha.

Groovy is not a new language, it was available years back and I used it for my project back in 2005/2006 because I require a dynamic language but still need to interact with Oracle via JDBC thin client. Other languages would require a thick client which I'm trying to avoid in a platform I have little control.

Grails was born out of the popularity of Rails, but well, yes Java is certainly evolving. The speculated successor is Scala, so you might just want to give it a look.
 

otacon

Arch-Supremacy Member
Joined
Apr 11, 2001
Messages
18,181
Reaction score
231
guys, i need a reliable n reasonably cheap email hosting for my friend. he only needs 1-2GB diskspace for abt 3 users now, maybe increasing to 5 or slightly more in future. i m looking at sparkstation, any bad comments abt them? i used them before last time, not much complains but i m a super light user, so cannot really comment much too

pls help
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ. Forum members and moderators are responsible for their own posts.

Please refer to our Community Guidelines and Standards, Terms of Service and Member T&Cs for more information.
Top