HWZ Forums

Login Register FAQ Mark Forums Read

Data Access Object pattern implementation: database and network

Reply
 
LinkBack Thread Tools
Old 05-12-2017, 01:14 PM   #1
Senior Member
 
Join Date: Jul 2009
Posts: 761
Data Access Object pattern implementation: database and network

Suppose I am writing a DAO to handle record retrieval from a database that is acting like a cache. It has a method getAllRecords().

CacheDAO::getAllRecords()

Now I also periodically need to refresh the cache subject to network availability i.e. if there is network connection available, I will try to fetch the latest data from a web source like XML, or JSON. If there is no network, I will simply grab whatever is already in the database cache. Let's call the method downloadRecords().

Option 1: Now, should I implement this downloadRecords() as a method inside the CacheDAO? CacheDAO will have 1 method accessing the database and 1 method fetching data from HTTP web source.
  1. CacheDAO::getAllRecords()
  2. CacheDAO::addRecord(...)
  3. CacheDAO::downloadRecords()

Option 2: Another option is to create a new DAO on its own: WebDAO, which somehow returns data from a HTTP source that are then inserted into CacheDAO database, followed by a call from CacheDAO.getAllRecords().
  1. CacheDAO::getAllRecords() // Read from database
  2. CacheDAO::addRecord(...) // Write to database
  3. WebDAO::downloadRecords() // Read from HTTP source

From my understanding is that a DAO implementation is tightly coupled to the storage system (e.g. database or web only) but not a mix of methods fetching data from different sources (database vs web service). This is to make it modular and flexible to swap in and out the DAO implementation class in case the underlying storage system (database, HTTP service) changes.

Comment?

Last edited by u0206397; 05-12-2017 at 01:33 PM.. Reason: Rename refreshRecords to downloadRecords for clarity
u0206397 is offline   Reply With Quote
Old 05-12-2017, 08:01 PM   #2
Supremacy Member
 
davidktw's Avatar
 
Join Date: Apr 2010
Posts: 9,679
Suppose I am writing a DAO to handle record retrieval from a database that is acting like a cache. It has a method getAllRecords().

CacheDAO::getAllRecords()

Now I also periodically need to refresh the cache subject to network availability i.e. if there is network connection available, I will try to fetch the latest data from a web source like XML, or JSON. If there is no network, I will simply grab whatever is already in the database cache. Let's call the method downloadRecords().

Option 1: Now, should I implement this downloadRecords() as a method inside the CacheDAO? CacheDAO will have 1 method accessing the database and 1 method fetching data from HTTP web source.
  1. CacheDAO::getAllRecords()
  2. CacheDAO::addRecord(...)
  3. CacheDAO::downloadRecords()

Option 2: Another option is to create a new DAO on its own: WebDAO, which somehow returns data from a HTTP source that are then inserted into CacheDAO database, followed by a call from CacheDAO.getAllRecords().
  1. CacheDAO::getAllRecords() // Read from database
  2. CacheDAO::addRecord(...) // Write to database
  3. WebDAO::downloadRecords() // Read from HTTP source

From my understanding is that a DAO implementation is tightly coupled to the storage system (e.g. database or web only) but not a mix of methods fetching data from different sources (database vs web service). This is to make it modular and flexible to swap in and out the DAO implementation class in case the underlying storage system (database, HTTP service) changes.

Comment?
DAO is just an OO concept to access a business object via a generic interface. It doesn't go further to dictate how you should perform your grouping. You can have one single DAO to interface across multiple different datasources as long as you feel the logical architecture make sense to you.

For example, I could have a DAO that says access Users. In a complex organisation which the Users may come from the LDAP, DB, FILE or a not well integrated SSO system. As far as to the application, the DAO provides the interface of retrieving a User based on the Identifier, or a group of Users based on Roles or other possible filter criteria.

In your case of caching, it can be incorporated into a specific DAO representing the data type you are retrieving. You can design your DAO to allow a caching component which can be implemented by various caching technologies such as in-memory storage such as Elasticsearch, MongoDB, or a local embedded database. If the caching module doesn't return the data, you then consult the datasource which can be an SSO, database and so forth.

Caching is a layer, it's not a DAO per sae. You can incorporate statically as a component of the DAO or you can implement it as a pluggable module that goes with any DAO you design. When I design DAO, I will segregate the DAO logically by the business concept such as USER, HISTORY, ACTION, JOURNAL, GROUP etc. For each DAO, you configure their individual caching approach, the retention of the cache, the data source and so forth.

If you want lets say your User DAO which can be adaptable to different datasource, you can create an abstract UserDAO class, and have subclasses of different adapters, like UserDAO_DB, UserDAO_FILE, UserDAO_LDAP, so that for the interface between the business modules to the DAO, the interface should be as uniform or similar as possible.

The mode of your caching is unique on its own. I often use such design so that the slowness of the backend will not affect the responsiveness of the frontend, since even if the data is stale, the users wouldn't know it, so I return stale data and if necessary, let the backend fetch the data asynchronously and then update the cache once the data is made available. All these without stopping the client response as long as the cache is filled.
davidktw is online now   Reply With Quote
Old 06-12-2017, 12:46 AM   #3
High Supremacy Member
 
Azzizz81's Avatar
 
Join Date: May 2001
Posts: 33,052
If it were me, I'll implement
Insert
Delete
Read
methods in the DAO

Create a service to refresh the DB table at a fixed timing if I can get data from the external source

I will avoid making a web service call in the DAO since it may violate the single responsibility principle

There are other ways to go about it like writing interceptors and aspects to do cache eviction on your interested methods but I don't see much value for the added complexity~
Azzizz81 is online now   Reply With Quote
Old 06-12-2017, 08:53 AM   #4
Senior Member
 
Join Date: Jul 2009
Posts: 761
Great answers from all. Thanks. I have a better idea now.
u0206397 is offline   Reply With Quote
Reply
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ. Forum members and moderators are responsible for their own posts.

Please refer to our Terms of Service for more information.


Thread Tools

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are On