Open Source Project

plasmic

Senior Member
Joined
Jun 7, 2010
Messages
618
Reaction score
0
Anyone interested to do a opensource project?

The whole ideal is have people with the same interest to come togather to do projects and have some fun.

My aim is not to commercial the project but if there is anyone interested to commercialise it, they are most welcome.

Do let me know in this post.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,301
Anyone interested to do a opensource project?

The whole ideal is have people with the same interest to come togather to do projects and have some fun.

My aim is not to commercial the project but if there is anyone interested to commercialise it, they are most welcome.

Do let me know in this post.

So what kind of project are you considering? Tell us more about the objectives.
 

plasmic

Senior Member
Joined
Jun 7, 2010
Messages
618
Reaction score
0
Weill I do not have any project in mind but will like to gather like minded people with the same interest. We can discuss as a team basis for what to do. Basically it is an aim to get togather and work on something the team will like. However, there is of course some set rules and goals which can be expanded further by the team. What i have in mind initially are:

1) Use only Opensource hw and sw
2) Project should impact our dialy life
3) Have an improvement in current life

So as an example only, maybe building a home intercom system using raspberry pi.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,301
Weill I do not have any project in mind but will like to gather like minded people with the same interest. We can discuss as a team basis for what to do. Basically it is an aim to get togather and work on something the team will like. However, there is of course some set rules and goals which can be expanded further by the team. What i have in mind initially are:

1) Use only Opensource hw and sw
2) Project should impact our dialy life
3) Have an improvement in current life

So as an example only, maybe building a home intercom system using raspberry pi.

Okay. Just some points for your consideration. Naturally opensource or not, it will be more effective and appealing if there is a head in the group whom already have an idea what to do. The collaboration will be easier to get started if one has spawn off an idea of what to begin with. It has always been in this manner. Very seldom people sit down together not knowing what to begin with.

It will be more fruitful if you have an idea what you want. Then you gather passionate people whom are interested with what you have proposed and work towards the objective. Perhaps a brief proposal will be a good start.
 

plasmic

Senior Member
Joined
Jun 7, 2010
Messages
618
Reaction score
0
Ok thanks.

I will come with a brief proposal.

Will you be keen?

If so do you have any specific you will like to do?
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,301
Ok thanks.

I will come with a brief proposal.

Will you be keen?

If so do you have any specific you will like to do?

Me ? lolx... plentiful. But remember who is the owner :) YOU ARE :)

You come up with the proposal and I will see if I'm interested, ok ? Remember it has to be something you want, not something others want. A snake can't move without its head.

Let me share a partial of my wish list to you, not in order of importance
1) Converting Joomla to use Object Based storage such as S3 or Ceph instead of Database.
2) Realtime 3D scanning using Kinect utilizing multicore systems
3) Microcontroller for iRobot like home cleaner (require 3D modelling + printing of parts). Modifying dump cleaner like Kärcher Robocleaner RC3000/RC4000
 

chenaz

Supremacy Member
Joined
Dec 21, 2011
Messages
9,780
Reaction score
370
working with strangers will have a lot conflicts and lack of determination and responsibility.
he may have the interest but lack of skills, so wat are u gonna say to him? he may have the skills but lack of responsibility. what are u gonna say to him as well? working as a team is not an easy job. just like a partnership business. u have ur thiking, he have his own thinking.
different objective or goal will be another issue.

u wan it to be open source but u mentioned commercial, means u might turn it into commercial . what if others does not agree on this?
how are u going to solve this?

everything have to map out first then decide on the path that u guys are going. having direction and rules will bring a team to work together else u will just like a headless flies
 
Last edited:

magpie4ever

Member
Joined
Jan 1, 2000
Messages
339
Reaction score
0
Me ? lolx... plentiful. But remember who is the owner :) YOU ARE :)

You come up with the proposal and I will see if I'm interested, ok ? Remember it has to be something you want, not something others want. A snake can't move without its head.

Let me share a partial of my wish list to you, not in order of importance
1) Converting Joomla to use Object Based storage such as S3 or Ceph instead of Database.
2) Realtime 3D scanning using Kinect utilizing multicore systems
3) Microcontroller for iRobot like home cleaner (require 3D modelling + printing of parts). Modifying dump cleaner like Kärcher Robocleaner RC3000/RC4000

Any progress on #1? I use Joomla at work and it definitely has a scaling problem. Sessions can be handed off to DynamoDB and database to RDS. S3 for user uploaded content is definitely the missing piece here.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,301
Any progress on #1? I use Joomla at work and it definitely has a scaling problem. Sessions can be handed off to DynamoDB and database to RDS. S3 for user uploaded content is definitely the missing piece here.

lolx. No R&D headroom for Joomla since I don't use it all the time in my company. Fortunately you can take a look at NuoDB. It has SQL layer over it's very scalable design for large scale storage design.

However it wouldn't replace MySQL or other database as-is. The SQL syntax in Joomla may need some transformation or tweaking to make them compatible with NuoDB SQL flavour.

Go read up and it and try out!!!

The problem with upgrading the storage layer of Joomla is not a technical problem, but rather a compatibility problem. This entry level CMS does not have sufficient abstraction in their db layer. It doesn't offer a wholesome db layer so that components written on top of it is database agnostic. End ups most sophisticated add-on to Joomla access the database directly instead of going through the db layer.

Even if you managed to convert the database layer to S3, you still have compatibility issue with components that you use with Joomla.
 
Last edited:

magpie4ever

Member
Joined
Jan 1, 2000
Messages
339
Reaction score
0
lolx. No R&D headroom for Joomla since I don't use it all the time in my company. Fortunately you can take a look at NuoDB. It has SQL layer over it's very scalable design for large scale storage design.

However it wouldn't replace MySQL or other database as-is. The SQL syntax in Joomla may need some transformation or tweaking to make them compatible with NuoDB SQL flavour.

Go read up and it and try out!!!

IMHO, RDS is a credible replacement for on-instance MySQL process. Leave AWS to figure out the scaling, patching, master slave replication across AZ. And no code change necessary. If I were writing a database connector for Joomla, I'd probably favour DynamoDB or SimpleDB (no upfront cost for database instances).

I think what's truly holding Joomla back is the lack of S3 support for filesystem requirements. For example, I can't run an autoscaling group of EC2 instances behind a load-balancer without a supporting Gluster / NFS setup. Or else an admin could find that a picture he added is only accessible some times, i.e. when the page is served from the instance he uploaded to.
 

davidktw

Arch-Supremacy Member
Joined
Apr 15, 2010
Messages
13,547
Reaction score
1,301
IMHO, RDS is a credible replacement for on-instance MySQL process. Leave AWS to figure out the scaling, patching, master slave replication across AZ. And no code change necessary. If I were writing a database connector for Joomla, I'd probably favour DynamoDB or SimpleDB (no upfront cost for database instances).

I think what's truly holding Joomla back is the lack of S3 support for filesystem requirements. For example, I can't run an autoscaling group of EC2 instances behind a load-balancer without a supporting Gluster / NFS setup. Or else an admin could find that a picture he added is only accessible some times, i.e. when the page is served from the instance he uploaded to.

Depending on how you feel scalability is, RDS is not necessarily in its entirety a solution to solve your scalability problem. It solve the fundamental database management, redundancy problem. It also simplified replication for administrators. With read replicas, it is possible for distributed read, but writing is still very much required on the master. Depending on the necessity in data consistency, application may require modification to design or consistency requirement when dealing with replication lag.

I wouldn't exactly call DynamoDB or SimpleDB a simple fix to scalable design. Mainly certain operations in Joomla may require transaction for some components. I don't remember seeing transactional requirement in common Joomla CMS operations yet. However some of the external components may. RDBMS normalized data storage allows only 1 copy of data to be updated when referenced across multiple records. While for document based storage solution, there is still a need to employ application based solution to ensure consistent data update, or using EMR(aka Map-Reduce technique) to efficiently update duplicated records in scalable manner.

Joomla is not exactly an Enterprise CMS so to speak. Having no S3 or object based storage solution is not a setback for CMS. For highly scalable ECMS design, it goes far more than just having which kind of storage. It's about the architecture that Joomla has that inhibit scalable design.

One of the ECMS product I have came across is scalable in architecture, yet doesn't require exceptional storage solution. In this design, it has publishing, versioning design into it's core concept. Having numerous frontend that are serving contents, and have one backend for editorial operations. Another frontend for preview purpose. Data consistency is achieved through versioning control and distributed updating of data from the backend to all the frontends. These frontends can be deployed all over the world and function without the backend to be online. Each of them have their own data storage using a RDBMS solution.

First of all, for CMS, the main bulk of load is normally serving content. Getting content from the public is much less. Of course, there are some web use cases where public submission is high. But that doesn't require everything to be perform inside the CMS. It can be always externally performed in separate solution that are designed for high load receiving data.

For your reason towards requiring a clustered filesystem or a consolidated filesystem to store uploaded data, this is totally up to your decision. You could also choose to have the file data written into the database and replicated across numerous instances, or you could also choose to have it written into the filesystem and having rsync to sync across other systems.

Obviously if you are looking for simple solution, it seems clustered filesystem is a good taste. I agree it is definitely easier to choose this approach and cope with the complexity in managing a clustered filesystem. But nothing is stopping you from shifting the consistency responsibility into the application layer design.
Ultimately it is a design choice. My point there are numerous approach to solve scalability issue. Sometimes the whole idea is knowing which is sacrifice and make up in different manner. Consistency, Availabilty, or Partition tolerance ?
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ Forums. Forum members and moderators are responsible for their own posts. Please refer to our Community Guidelines and Standards and Terms and Conditions for more information.
Top