Do u all belive in bazi / zi wei dou shu? - Part 13

dannytan87

dannytan87

Banned
Joined
Oct 3, 2011
Messages
56,886
Reaction score
2,453
CyberTan said:
Through SQL and Python right?



Posted with HardwareZone Forums app
Click to expand...

thats just tools, and a very small part of it.........

SQL is used for data extraction............

data engineering has it call ETL extract transform load.........

so like extract all female patron between 2019 2020 sale record...... transform into a standard format and load into the pipeline.............

pipeline will then process.......

SQL can be use in spark jobs...... spark support python java and something i forget 3 languages......

so maybe they can store in hadoop using query engine like pig or hive(has SQL like query)..... code can connect to hadoop using hive or pig query....... extract all the data and then stream each record to maybe like kafka...... spark job will literally take each record and broadcast on a messaging system like kafka.....

there are many ways and many options.... each have pros and cons for different scenario and use case...... maybe the data analysis is a time series..... a weekly data report...... then will have some set of tools and framework..... maybe some data is large image dataset then will have another suitable set of tools and framework...........

------------------

SQL SEQUEL is structured querys...... there maybe unstructure, semi structure data..... some need data consistency, some need high availability, some need partition........ then will have different database....... HBase, Cassandra, mongol, relational DB.... all store records tuples different and may or maynot use SQL.......

some maybe use data partitioning, some may use data shard...... to keep on different database servers..... then have some kind of registry and cluster to serve different querys...........
 
New Eden

New Eden

High Supremacy Member
Joined
Jul 10, 2015
Messages
31,883
Reaction score
1,150
pepsitwist said:
This seems to apply to anywhere also? See heng suay.

True is more potential to grow, but no work life balance one, especially the higher you climb. My email forever synced to phone, go medical/hospitalisation leave also still responding/attending calls on phone. Last time go overseas holiday also got people still contacting me.

Unlike I see my other friend... in CS on leave nobody will ka jiao you. Weekends also no need to work one.
Click to expand...

Same la.. im just about one man show.. very sian sometimes. One can only take so much responsibilities.

At least your pay surely very high. Mine is.... :s22:
 
Kuudere

Kuudere

High Supremacy Member
Joined
Nov 4, 2016
Messages
25,636
Reaction score
16,783
dannytan87 said:
thats just tools, and a very small part of it.........

SQL is used for data extraction............

data engineering has it call ETL extract transform load.........

so like extract all female patron between 2019 2020 sale record...... transform into a standard format and load into the pipeline.............

pipeline will then process.......

SQL can be use in spark jobs...... spark support python java and something i forget 3 languages......

so maybe they can store in hadoop using query engine like pig or hive(has SQL like query)..... code can connect to hadoop using hive or pig query....... extract all the data and then stream each record to maybe like kafka...... spark job will literally take each record and broadcast on a messaging system like kafka.....

there are many ways and many options.... each have pros and cons for different scenario and use case...... maybe the data analysis is a time series..... a weekly data report...... then will have some set of tools and framework..... maybe some data is large image dataset then will have another suitable set of tools and framework...........

------------------

SQL SEQUEL is structured querys...... there maybe unstructure, semi structure data..... some need data consistency, some need high availability, some need partition........ then will have different database....... HBase, Cassandra, mongol, relational DB.... all store records tuples different and may or maynot use SQL.......

some maybe use data partitioning, some may use data shard...... to keep on different database servers..... then have some kind of registry and cluster to serve different querys...........
Click to expand...

You must be DR for bazi, same as me
 
HomeDaddy

HomeDaddy

Suspended
Joined
Sep 17, 2001
Messages
7,761
Reaction score
304
billovee said:
So sad, these Chinese verse suit me so well
Click to expand...

Everything is in the mind.

要有雄心壮志，还是小富即安。

每个人追求的东西不一样。没有对错。

With current life expectancies, u could start out at 50 and still achieve decent success by 60. Its never too late.
 
aiveee

aiveee

Senior Member
Joined
May 30, 2013
Messages
2,318
Reaction score
19
Wa hahah thank you!!

But who is TUV?

Anyway hi everyone!

New Eden said:
:s13:

Mostly chit chat these days.

If you want your bazi to be read, can pm TUV with your dob, and gender. Tell him if you can read mandarin or not.

Remember to clear your inbox, dont make him type a lot then realise cannot send to you 🤣

If want to read zwds, can pm dogz with your dob.

:s12:
Click to expand...
 
billovee

billovee

High Supremacy Member
Joined
Feb 8, 2005
Messages
40,127
Reaction score
548
feng1314 said:
Everything is in the mind.

要有雄心壮志，还是小富即安。

每个人追求的东西不一样。没有对错。

With current life expectancies, u could start out at 50 and still achieve decent success by 60. Its never too late.
Click to expand...

Thanks for encouragement,

Ice cream seller will always be ice cream seller
 
luohongxian

luohongxian

Master Member
Joined
Dec 6, 2018
Messages
2,984
Reaction score
881
feng1314 said:
Everything is in the mind.

要有雄心壮志，还是小富即安。

每个人追求的东西不一样。没有对错。

With current life expectancies, u could start out at 50 and still achieve decent success by 60. Its never too late.
Click to expand...


Provided by 50 one still got good health. No health can forget abt achieving success
 
You must log in or register to reply here.
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ. Forum members and moderators are responsible for their own posts.

Please refer to our Community Guidelines and Standards, Terms of Service and Member T&Cs for more information.
Top