RSS feed [root] /design /weblog




login:

password:

title search:




 


Sat Mar 02 17:01:24 HKT 2013

design



(google search) (amazon search)
second
download zip of files only

Tue Apr 23 23:31:01 HKT 2013 From /weblog/design/concurrency

thread


Stealing thread - http://badamczewski.blogspot.com.au/2012/05/work-stealing.html

Intel Guide for Developing Multithreaded Applications - http://software.intel.com[..]or-developing-multithreaded-applications

Difference ways to stop a thread - http://www.ddj.com[..]ept_url=/hpc-high-performance-computing/

Interesting, I am not sure if I agree, but chrome ( which use fork ) are really cool in performance:
There’s another problem with Unix programming in Ruby that I’ll just touch on briefly: Java people and Windows people. They’re going to tell you that fork(2) is bad because they don’t have it on their platform, or it sucks on their platform, or whatever, but it’s cool, you know, because they have native threads, and threads are like, way better anyways.

Fuck that.

Don’t ever let anyone tell you that fork(2) is bad. Thirty years from now, there will still be a fork(2) and a pipe(2) and a exec(2) and smart people will still be using them to solve hard problems reliably and predictably, just like they were thirty years ago.

MRI Ruby people need to accept, like Python (you have seen multiprocessing, yes?), that Unix processes are one of two techniques for achieving reliable concurrency and parallelism in server applications. Threads are out. You can use processes, or async/events, or both processes and async/events, but definitely not threads. Threads are out.
http://tomayko.com/writings/unicorn-is-unix

1x1 win M*N - http://binkley.blogspot.com/2012/01/1-1-beats-n-m.html

Best number of threads:
N = number of CPUs
U = target CPU utilization (0 <= U <= 1)
W/C = ration of wait time to cpu time (measured through profiling)

threads = N * U * (1 + W/C) - http://www.ibm.com/developerworks/library/j-jtp0730.html http://stackoverflow.com[..]ratio-of-java-threads-to-cpus-on-solaris

Threads Basics - http://www.hpl.hp.com/techreports/2009/HPL-2009-259html.html http://www.hpl.hp.com/personal/Hans_Boehm/c++mm/threadsintro.html

(google search) (amazon search)


Fri Apr 05 17:31:38 HKT 2013 From /weblog/design/concurrency

concurrency


How to write safer concurrency code - http://www.artima.com/forums/flat.jsp?forum=276&thread=178345

reentrant and thread safe functions - http://kevinrodrigues.com[..]/31/reentrant-and-thread-safe-functions/

Libraries / toolkits for multicore process - http://www.ddj.com[..]intableArticle.jhtml?articleID=212900103

Introduction - http://www.ddj.com[..]QQSNDLRSKHSCJUNN2JVN?articleID=212903586 http://www.ddj.com[..]CQSNDLRSKHSCJUNN2JVN?articleID=213001517

Collections of links - http://dobbscodetalk.com[..]rallel-Or-Get-Left-Behind.html&Itemid=29

Briefing of difference modeling of threading system - http://www.ddj.com[..]intableArticle.jhtml?articleID=215900465 http://software.intel.com[..]inners-guide-to-multithreaded-libraries/

http://natishalom.typepad.com[..]haloms_blog/2010/08/concurrency-101.html

Saving the Failwhale: The Art of Concurrency (Page last updated December 2012, Added 2012-12-26, Author Dhanji R. Prasanna, Publisher informit). Tips:

1) Contention is unavoidable - some resources are just slower, and you must wait for them. The secrets to good concurrency are 1) ensuring that these slower resources are rarely used, and 2) during such waiting periods, giving the faster tiers other work to do so that they continue to be utilized well.
2) Overuse of synchronization constructs such as locks and mutexes leads to systems that perform poorly under load.
3) ConcurrentHashMap is an efficient thread-safe map while HashMap is not thread-safe.
4) ConcurrentHashMap doesn't do away with locks, it still uses them but it uses more than the single global lock, so that threads gain some measure of concurrency. It uses separate locks for partitions, so that multiple threads writing to the map are likely to access different partitions, using separate locks and therefore process their data simultaneously. This technique is known as lock-striping. Efficient striping uses a number of locks proportional to the number of CPU cores in a system.
5) The asynchronous processing model smooths resource spikes by adding requests to a queue which is serviced by a pool of workers - spikes in requests make the queue grow rather than overloading the workers. (The ExecutorService is essentially a thread pool accompanied by a task queue.)

http://www.informit.com/articles/article.aspx?p=1994789

Discussion of using difference model for concurrency - http://highscalability.com[..]cks-application-architecture-pros-a.html

(google search) (amazon search)


Fri Apr 05 17:30:10 HKT 2013 From /weblog/design/distribute

amazon


Tutorial of how to use EC2 - http://www.thediscoblog.com/2011/09/15/working-with-ec2-video/

http://www.dodgycoder.net[..]ability-lessons-from-google-youtube.html

(google search) (amazon search)


Wed Apr 03 16:29:59 HKT 2013 From /weblog/design

Implementation


Tired with duplicating Try-Catch for resource acquisition? Can try to create proxy method - http://hamletdarcy.blogspot.com[..]clarative-synchronization-with-java.html

Discussion of naming of if block - http://blog.objectmentor.com/articles/2008/09/26/if-methods-redux

Factors for choosing a strategic technology stack, runtime performance is not a factor for him, probably good idea - http://jchyip.blogspot.com[..]1/04/factors-for-choosing-strategic.html

Don't think , stay focus - http://www.threeriversinstitute.org/blog/?p=594

Using class name as variable name is a bad thing? I doubt about it, but interesting idea - http://www.warski.org/blog/2013/01/dry-parameter-names/

The problem of using generic name - http://martinfowler.com/bliki/TransparentCompilation.html

(google search) (amazon search)


Mon Mar 25 10:34:15 HKT 2013 From /weblog/design/distribute

nosql


Compare SQL ( RDBMS ) and noSQL ( object base {distributed??} ) data management - http://queue.acm.org/detail.cfm?id=1961297&ref=fullrss

The definition - http://martinfowler.com/bliki/NosqlDefinition.html http://martinfowler.com/articles/nosqlKeyPoints.html

Is Cassandra really that good? - http://www.datastax.com/dev/blog/2012-in-review-performance

http://martinfowler.com/articles/bigData/

NoSQL Options Compared - http://www.drdobbs.com[..]cleId=240151198&siteSectionName=database

A 3 Step Guide to Getting Started with NoSQL - http://jyops.blogspot.com.au[..]-step-guide-to-getting-started-with.html

(google search) (amazon search)


Fri Mar 08 08:50:44 HKT 2013 From /weblog/design

API design guideline


This is a message from a management blog, but I think the arguement is also apply API design. In fact, I think most critical difference of good and bad API is knowing which small detail is important and which is not - http://www.goodproductmanager.com[..]er.com/2007/11/08/sweat-the-small-stuff/

Design tips:
http://www.artima.com/weblogs/viewpost.jsp?thread=142428
http://openide.netbeans.org/tutorial/api-design.html
http://www.cincomsmalltalk.com[..]gView?showComments=true&entry=3258158706
http://today.java.net[..]its-of-highly-profitable-developers.html
http://www.infoq.com/news/2007/08/why-api-design-matters
http://www.infoq.com/presentations/effective-api-design
http://neuroning.com/2006/11/19/on-api-design-guidelines

About Compatibility issues
http://wiki.eclipse.org/Evolving_Java-based_APIs

A blog of using interfaces-vs-abstract-classes
http://hoskinator.blogspot.com[..]6/04/interfaces-vs-abstract-classes.html

XOM design overview - http://www.xom.nu/designprinciples.xhtml#d0e309

You need to identify the business value but not pick any tools/design just because it is cool - http://myarch.com/what-is-good-soa

A lot of links here - http://kasparov.skife.org/blog-live/src/api-design-refs.writeback http://discuss.joelonsoftware.com/default.asp?design.4.527465

An introduction of good OSS project to study their source - http://techkriti.wordpress.com[..]com/2007/06/28/learning-from-the-source/

CCCCDPIPE - http://blog.objectmentor.com/articles/2007/08/02/which-came-first

Discussion of why getting feedback quick is important - http://blog.objectmentor.com[..]u-dont-know-until-you-take-the-next-step

Someone saying that routines is the greatest invention in CS, I agree - http://www.codinghorror.com/blog/archives/001129.html

New way of modeling system then layering, The Onion Architecture - http://jeffreypalermo.com/blog/the-onion-architecture-part-1/

SOLID - http://www.lostechies.com[..]of-the-month-march-solid-principles.aspx http://dotnet.dzone.com/news/solid-software-works <-- very good picture! http://www.blackwasp.co.uk/SOLIDPrinciples.aspx

Idempotent, Orthogonality, Immutability - http://www.codinghorror.com/blog/archives/001244.html

Some example of how to driven good API - http://jdegoes.squarespace.com[..]al/2009/5/11/good-api-design-part-3.html

It's not enough to write tests for an API you develop, you have to write unit tests for code that uses your API. When you do, you learn first-hand the hurdles that your users will have to overcome when they try to test their code independently. http://butunclebob.com[..]MichaelFeathers.TheGoldenRuleOfApiDesign

Active interface vs. Passive interface, for me this is another way to under "Tell, don't ask" - http://code-o-matic.blogspot.com[..]-on-api-design-call-stack-as-source.html

The Principles of Good Programming - http://www.artima.com/weblogs/viewpost.jsp?thread=331531

http://net.tutsplus.com[..]software-principles-you-must-understand/

Difference choice of how API evolute with change - http://blog.jooq.org[..]sive-api-evolution-with-java-interfaces/

JUnit's evolving structure. - http://edmundkirwan.com/general/junit.html

(google search) (amazon search)


Sat Mar 02 17:01:24 HKT 2013 From /weblog/design

code metric


Contain resource of measurable code metric information - http://thediscoblog.com[..]20/code-quality-for-software-architects/

Meaningful Metrics - http://www.developsense.com/2009/01/meaningful-metrics.html

http://programmers.stackexchange.com[..]ions/115839/how-clean-should-new-code-be

Guidelines for a more appropriate use of metrics

Explicitly link metrics to goals
Favor tracking trends over absolute numbers
Use shorter tracking periods
Change metrics when they stop driving change

http://martinfowler.com/articles/useOfMetrics.html

(google search) (amazon search)


Thu Feb 14 12:33:33 HKT 2013 From /weblog/design/pattern

singleton


How to get ride of it - http://tech.puredanger.com/2007/07/03/pattern-hate-singleton/

Trying to make singleton testable - http://alexruiz.developerblogs.com/?p=1832

Why singleton? Some time may be just create the object once at the beginning is fine - http://www.butunclebob.com[..]ticleS.UncleBob.SingletonVsJustCreateOne

And a good article of telling why don't use singleton - http://www-106.ibm.com[..]works/webservices/library/co-single.html

About how to get it right in Java - http://idiotechie.com/?p=1256

(google search) (amazon search)


Sun Feb 03 11:10:31 HKT 2013 From /weblog/design/interview

Jim Showalter


Principal engineer at Intuit - http://www.javaworld.com[..]im-showalter-of-intuit.html&site=jw_core

(google search) (amazon search)


Thu Jan 31 23:26:25 HKT 2013 From /weblog/design/distribute

scalability


There are two key primary ways of scaling web applications which is in practice today.
1) “Vertical Scalability” - Adding resource within the same logical unit to increase capacity. An example of this would be to add CPUs to an existing server, or expanding storage by adding hard drive on an existing RAID/SAN storage.
2) “Horizontal Scalability” - Adding multiple logical units of resources and making them work as a single unit. Most clustering solutions, distributed file systems, load-balancers help you with horizontal scalability.

Scalability can be further sub-classified based on the “scalability factor”.
1) If the scalability factor stays constant as you scale. This is called “linear scalability“.
2) But chances are that some components may not scale as well as others. A scalability factor below 1.0 is called “sub-linear scalability“.
3) Though rare, its possible to get better performance (scalability factor) just by adding more components (i/o across multiple disk spindles in a RAID gets better with more spindles). This is called “supra-linear scalability“.
4) If the application is not designed for scalability, its possible that things can actually get worse as it scales. This is called “negative scalability“.

http://www.royans.net/arch/2007/09/22/what-is-scalability/

Report of building web application with 55k pageload with rail - http://shanti.railsblog.com[..]mongrels-handled-a-550k-pageview-digging

XMPP a IM protocol about scalability - http://www.process-one.net[..]icle/the_aol_xmpp_scalability_challenge/

Presentation and resources of making you website more scalable - http://www.scribd.com[..]9/Real-World-Web-Performance-Scalability http://www.theserverside.com[..]lications&asrc=EM_NLN_3990118&uid=703565 http://www.theserverside.com[..]ionsPart2&asrc=EM_NLN_3990119&uid=703565

Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system:
* The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency.
* Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool.
* Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability.
* Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter.
* Forgetting to check the time. To properly scale a system it is imperative to manage the time alloted for requests to be handled.
* Hero Pattern. One popular solution to the operation issue is a Hero who can and often will manage the bulk of the operational needs. For a large system of many components this approach does not scale, yet it is one of the most frequently-deployed solutions.
* Not automating. A system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome.
* Monitoring. Monitoring, like testing, is often one of the first items sacrificed when time is tight.

http://highscalability.com/scalability-worst-practices

Useful Corporate Blogs that Talk About Scalability - http://highscalability.com[..]l-corporate-blogs-talk-about-scalability

Overview of mapreduce and how it compare with other distributed programming model -http://natishalom.typepad.com[..]0/is-mapreduce-going-to-main-stream.html

Paper of data store at amazon http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

Discuss how haven't sync can cause performance issue - http://www.theserverside.com[..]lications&asrc=EM_NLN_6273194&uid=703565 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6423457

Discussion about Cloud Based Memory Architectures - http://highscalability.com[..]ased-memory-architectures-next-big-thing

http://highscalability.com[..]alability-and-performance-best-practices

Interview with google engineer - http://www.zdnet.co.uk[..]gle-at-scale-everything-breaks-40093061/

Surprisingly youtube is blocking - http://highscalability.com[..]e-scalability-lessons-in-30-minutes.html

If we are seeing a sustained arrival rate of requests, greater than our system is capable of processing, then something has to give. Having the entire system degrade is not the ideal service we want to give our customers. A better approach would be to process transactions at our systems maximum possible throughput rate, while maintaining a good response time, and rejecting requests above this arrival rate. - http://mechanical-sympathy.blogspot.com.au[..]apply-back-pressure-when-overloaded.html

How twitter scaling - http://www.infoq.com/presentations/Twitter-Analytics

(google search) (amazon search)


Tue Jan 29 10:35:26 HKT 2013 From /weblog/design

High level software design discussion


A nice table:
Do Don't
1. Focus on interfaces. 2. Be worried about the implementation.
3. Mention semantic contract using interfaces. 4. Let subtypes break this semantic contract of their parent types.
5. Service decoupling using interfaces. 6. Be coupled with specific concrete implementation.
7. Family extension using interfaces. 8. Break the family relationship using concrete implementation.
9. Establish a family rule by abstract classes. 10. Impose your own rule through concrete classes.
11. Let interfaces answer all "what" about the system. 12. Forget to mention "how" are you answering all "what" from interfaces.
Also a good introduction of Strategy pattern

http://today.java.net[..]/03/28/testing-java-object-oriented.html How to determine the levels of abstraction - http://programmers.stackexchange.com[..]w-to-determine-the-levels-of-abstraction Simple and clear, good design -> Correct, clear, easy to change - http://randomartifacts.blogspot.com.au[..]rrectness-clarity-and-changeability.html
(google search) (amazon search)


Mon Jan 28 21:31:57 HKT 2013 From /weblog/design

classic


List of Turing Award Lecture - http://c2.com/cgi/wiki?TuringAwardLecture

classic-computer-science-text - http://mark-watson.blogspot.com[..]0/classic-computer-science-text-and.html

How emacs is beautiful - http://martinfowler.com/bliki/InternalReprogrammability.html

(google search) (amazon search)


Mon Jan 21 17:07:23 HKT 2013 From /weblog/design

model


Agile Models Distilled: Potential Artifacts for Agile Modeling - http://www.agilemodeling.com/artifacts/

Difference between Association, Composition, Aggregation, Dependency, Abstraction, Realization, Generalization - http://techie-experience.blogspot.gr[..]1/quick-summary-object-associations.html

(google search) (amazon search)


Sun Jan 20 01:35:15 HKT 2013 From /weblog/design

DataStructure


"Bad programmers worry about the code. Good programmers worry about data structures and their relationships." - http://programmers.stackexchange.com[..]uote-about-good-programmer/163187#163187

Compare between schema and schema=less ( like key value mapping ) data - http://martinfowler.com/articles/schemaless/

(google search) (amazon search)


Sun Jan 06 18:17:22 HKT 2013 From /weblog/design/distribute

distribute


In one sentence, here's why: humans are notoriously bad at keeping "self" distinct from "other". Egomania, projection (transference), and enmeshment are well-known symptoms of this problem. OK, so I hear you saying, "yeah, but what does this have to do with programming?" It certainly seems absurd to suggest that if we are bad at something we know the most about (our "selves"), how could we possibly say that we have a good approach for the programming analogues - objects, modules, etc. - http://www.artima.com/weblogs/viewpost.jsp?thread=46706

Argue why space base design is better than n-tier design - http://www.google.com[..]0The%20End%20of%20Tier-based%20Computing

Some key research of google for distributed computation - http://www.infoq.com/news/2007/06/google-scalability

Someone think we are not yet (per Oct 2007) have good language support for distibuted computing - http://kasparov.skife.org/blog/2007/10/11/

A blog contain a lot distributed computing information - http://www.highscalability.com/

How Wikipedia manage their site - http://dammit.lt/uc/workbook2007.pdf

Google tutorial for Design Distributed System - http://code.google.com/edu/parallel/dsd-tutorial.html

http://en.wikipedia.org/wiki/Distributed_hash_table

The Hadoop Distributed File System: Architecture and Design - http://hadoop.apache.org/core/docs/r0.18.0/hdfs_design.html

http://www.metabrew.com[..]-a-list-of-distributed-key-value-stores/

Applying experience from CPU design for distributed solution - http://blog.paralleluniverse.co[..]o/post/26909672264/on-distributed-memory

(google search) (amazon search)


Sun Jan 06 17:43:00 HKT 2013 From /weblog/design/exception

suggestion


With nice examples - http://www.javaspecialists.eu/archive/Issue162.html

Try to do better working than log and quit - http://blog.objectmentor.com[..]9/01/05/abstracting-away-from-exceptions

http://www.makinggoodsoftware.com[..]ceptions-in-java-4-basic-considerations/

Exception Patterns - http://c2.com/cgi/wiki?ExceptionPatterns

The Scourge of Error Handling, "go" experience - http://www.drdobbs.com[..]5_h&elq=2c4502fb959647fd9ad16dc5c0f7d75d

(google search) (amazon search)


Sat Jan 05 11:19:18 HKT 2013 From /weblog/design/concurrency

sharing


false-sharing - http://www.1024cores.net[..]ck-free-algorithms/false-sharing---false

(google search) (amazon search)


Thu Jan 03 22:18:21 HKT 2013 From /weblog/design/concurrency

issues



Case study of a parallel slowdown, and the analysis of how to find it out - http://webtide.intalio.com[..]2/avoiding-parallel-slowdown-in-jetty-9/

(google search) (amazon search)


Wed Jan 02 23:25:22 HKT 2013 From /weblog/design

performance


http://highscalability.com[..]6/big-list-of-20-common-bottlenecks.html

Nothing specific, mostly code clean up, refactoring and simplification, the performance boost was a surprise. <- This is a good one - http://bad-concurrency.blogspot.com.au[..]12/07/disruptor-v3-faster-hopefully.html

Is there anything we can do about this when designing algorithms and data-structures? Yes there is a lot we can do. If we perform chunks of work on data that is co-located, and we stride around memory in a predictable fashion, then our algorithms can be many times faster. For example rather than using bucket and chain hash tables, like in the JDK, we can employ hash tables using open-addressing with linear-probing. Rather than using linked-lists or trees with single items in each node, we can store an array of many items in each node. - http://mechanical-sympathy.blogspot.com.au[..]emory-access-patterns-are-important.html

Skip lists are used instead of b-trees because b-trees don’t scale. - http://highscalability.com[..]ast-mvcc-inmem-lockfree-codegen-and.html

Beware about the performance issue of static initalization - http://stackoverflow.com[..]-cause-problems-with-multi-threading-how

(google search) (amazon search)


Tue Jan 01 18:37:57 HKT 2013 From /weblog/design

memory


It won't happened at java, but for non-GC language like C++, crash at constructor will cause memory leak: http://blogs.asman-it.com.au[..]oders-have-it-easy-crashing-constructors

Discussion of if there are way to recover OOME - http://dobbscodetalk.com[..]-a-Non-Recoverable-Error-.html&Itemid=29

http://blog.maxindelicato.com[..]ghly-available-inmemory-shard-index.html

Discussion about using memory as store - http://martinfowler.com/bliki/MemoryImage.html

When using direct memory can be faster, when you have a large amounts of data and/or you have to perform some IO with that data. Tick data is a good example - http://vanillajava.blogspot.com.au[..]n-using-direct-memory-can-be-faster.html

(google search) (amazon search)


Tue Jan 01 18:03:07 HKT 2013 From /weblog/design/concurrency

queue


Fast synchronization between a single producer and single consumer - http://www.bluebytesoftware.com[..]eenASingleProducerAndSingleConsumer.aspx

Lock free queue idea. - http://groups.google.com[..]read/thread/82066179448783da?hl=en&pli=1

Use of Disruptor - http://www.infoq.com/presentations/Disruptor

Use of Queue - http://highscalability.com[..]ble-presents-queue-er-message-queue.html

(google search) (amazon search)


Tue Jan 01 10:28:13 HKT 2013 From /weblog/design

quote


Quotations on simplicity in software design - http://www.jbox.dk/quotations.htm

2 hard things, caching and naming - http://martinfowler.com/bliki/TwoHardThings.html


(google search) (amazon search)


Tue Dec 18 00:49:34 HKT 2012 From /weblog/design/interview

Alan Kay


Not interview, but a lot of useful link - http://www.codinghorror.com/blog/archives/001213.html

http://www.drdobbs.com[..]0_h&elq=c2568ee4975f4e69b244d72a9a8e2487

(google search) (amazon search)


Tue Oct 02 20:34:57 HKT 2012 From /weblog/design/distribute

spanner


Google spanner

https://cloudant.com/blog/cloudant-labs-on-google-spanner/
http://nosql.mypopescu.com[..]ogle-spanners?e57078b0?317bb970?b3b99980

(google search) (amazon search)


Mon Oct 01 23:49:18 HKT 2012 From /weblog/design

constructor


Examples of proper and inproper constructor - http://misko.hevery.com[..]s-guide/flaw-constructor-does-real-work/

(google search) (amazon search)