tag:blogger.com,1999:blog-58948463303419131762023-06-20T05:59:59.210-07:00Noooooo...SQL ...!!Vishnu Chilamakuruhttp://www.blogger.com/profile/13099618632034328290noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-5894846330341913176.post-34918878946352555252016-02-29T07:01:00.000-08:002016-02-29T07:03:43.208-08:00Three Database Revolutions<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="text-align: justify;">There are three database revolutions that happened so far. </span><br />
<div>
<ul style="text-align: left;">
<li style="text-align: justify;">The first revolution was driven by the emergence of the electronic computer.</li>
<li style="text-align: justify;">The second revolution by the emergence of the relational database.</li>
<li style="text-align: justify;">The third revolution has resulted in an explosion of non-relational database alternatives driven by the demands of modern applications that require global scope and continuous availability.</li>
</ul>
<div style="text-align: justify;">
Lets have a look on these three waves of database technologies and discuss the market and technology forces leading to today’s next generation databases.</div>
</div>
<div>
<br /></div>
<div>
<b>1950-1972 (Pre - Relational)</b></div>
<div>
<ul style="text-align: left;">
<li>1951 - Magnetic Tape</li>
<li>1952 - Magnetic Disk</li>
<li>1961 - ISAM</li>
<li>1965 - Hierarchical Model</li>
<li>1968 - IMS</li>
<li>1969 - Network Model</li>
<li>1971 - IDMS</li>
</ul>
<b>1972 - 2005 (Relational)</b></div>
<div>
<ul style="text-align: left;">
<li>1970 - Codd's Paper</li>
<li>1974 - System R</li>
<li>1978 - Oracle</li>
<li>1980 - Commercial Ingres</li>
<li>1981 - Informix</li>
<li>1984 - DB2</li>
<li>1987 - Sybase</li>
<li>1989 - Postgres</li>
<li>1989 - SQL Server</li>
<li>1995 - MySQL</li>
</ul>
<b>2005 - 2015 ( The Next Generation) </b></div>
<div>
<ul style="text-align: left;">
<li>2003 - MarkLogic</li>
<li>2004 - MapReduce</li>
<li>2005 - Hadoop</li>
<li>2005 - Vertica</li>
<li>2007 - Dynamo</li>
<li>2007 - Neo4J</li>
<li>2008 - Cassandra</li>
<li>2008 - HBase</li>
<li>2008 - NuoDB</li>
<li>2009 - MongoDB</li>
<li>2010 - VoltDB</li>
<li>2010 - Hana</li>
<li>2011 - Riak</li>
<li>2012 - Aerospike</li>
<li>2014 - Spile Machine</li>
</ul>
<div>
<div style="text-align: justify;">
In the 20 years following the widespread adoption of electronic computers, a range of increasingly sophisticated database systems emerged.emerged. Shortly after the definition of the relational model in 1970, almost every significant database system shared a common architecture. The three pillars of this architecture were the relational model, ACID transactions, and the SQL language.</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<div>
However, starting around 2008, an explosion of new database systems occurred, and none of these</div>
<div>
adhered to the traditional relational implementations. Few of the main reasons which led to the new generation of databases is because of 3 V's.</div>
<div>
<ul>
<li>Volume - High Volume of Data. Ex: MB, GB, TB, Petabytes of data. </li>
<li>Velocity - High Velocity of Data. Ex: Batch, Periodic, Near Real Time, Real Time</li>
<li>Variety - More Variety of Data. Ex: Database, Web, Photo, Audio, Video, Unstructured</li>
</ul>
<div>
In the next blog posts we will go through few of the popular Next Generation Databases, their overall architectures and use cases where they will be used.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
</div>
</div>
</div>
</div>
Vishnu Chilamakuruhttp://www.blogger.com/profile/13099618632034328290noreply@blogger.com0tag:blogger.com,1999:blog-5894846330341913176.post-51296845864221370312013-01-29T22:31:00.002-08:002016-02-29T07:01:47.813-08:00How MongoDB survives From SQL or Query Injection<div dir="ltr" style="text-align: left;" trbidi="on">
As We know <a href="http://en.wikipedia.org/wiki/SQL_injection">SQL injection</a> is one of the most famous way people try to hack the SQL based applications.I came to know about interesting thing how <a href="http://en.wikipedia.org/wiki/MongoDB">MongoDB</a> survives from this SQL injection while reading the mongodb docs.<br />
<div>
<br /></div>
<div>
For SQL based applications most of the drivers support accessing SQL data using query as String which makes the access vulnerable.</div>
<div>
For Example in Java we use to get the data from SQL as follows,</div>
<div>
<br /></div>
<div>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-29372173-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script><br />
<div>
String query = "SELECT ZipCode,State FROM zipcodes WHERE City = '+city+' AND State = '+state+'";</div>
</div>
<div>
<div>
connection = DriverManager.getConnection(jdbcurl, username, password);</div>
<div>
Statement stmt = connection.createStatement();</div>
</div>
<div>
ResultSet rs = stmt.executeQuery(query);</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
In case of MongoDB there is no vulnerability because all the drivers creates a BSON object for the given Query instead of calling the DB as a string itself.</div>
<div>
<br /></div>
<div>
For MongoDb in Java QueryBuilder is used to build Queries for accesing MongoDb Data,</div>
<div>
<br /></div>
<div>
<div>
DBObject query = QueryBuilder.start("City").is(city).and("State").is(state).get();</div>
</div>
<div>
<br /></div>
<div>
<span style="background-color: white; font-family: "helvetica" , "arial" , sans-serif; font-size: 16px; line-height: 1.5em;">As a client program assembles a query in MongoDB, it builds a BSON object, not a string. Thus traditional SQL injection attacks are not a problem. </span></div>
<div>
<div style="background-color: white; font-family: helvetica, arial, sans-serif; font-size: 16px; line-height: 1.5em; text-align: -webkit-auto;">
MongoDB represents queries as BSON objects. Typically client libraries provide a convenient, injection free, process to build these objects.</div>
</div>
</div>
Vishnu Chilamakuruhttp://www.blogger.com/profile/13099618632034328290noreply@blogger.com2tag:blogger.com,1999:blog-5894846330341913176.post-47900425423745409342012-12-14T22:53:00.001-08:002016-02-29T07:02:11.510-08:00Comparison of Popular NoSql databases (MongoDb,CouchDb,Hbase,Neo4j,Cassandra)<div dir="ltr" style="text-align: left;" trbidi="on">
There are many SQL databases so far.But i personally feel the 15 years history of SQL coming to an end as everyone is moving to an era of <a href="http://en.wikipedia.org/wiki/Big_data">BigData</a>. As experts say SQL databases are not a best fit for Big Data No Sql databases came into picture as a best fit for this which provides more flexibility in storing data.<br />
I just want to compare few popular NoSql databases that are available at this point of time.Few well known NoSql databases are<br />
<ul style="text-align: left;">
<li><a href="http://www.mongodb.org/">MongoDb</a></li>
<li><a href="http://cassandra.apache.org/">Cassandra</a></li>
<li><a href="http://hbase.apache.org/">Hbase</a></li>
<li><a href="http://couchdb.apache.org/">CouchDb</a></li>
<li><a href="http://neo4j.org/">Neo4j</a></li>
</ul>
NoSql databases differ each other more than the way Sql databases differ from each other.I think its one's responsibility to choose the appropriate NoSql database for their application based on their use case.Lets do a quick comparison of these databases.<br />
<br />
<span style="color: #20124d;">MongoDb</span><br />
<br />
<ul style="text-align: left;">
<li><b>Written in :</b> c++</li>
<li><b>Main point :</b> Retains some friendly properties of SQL (Query, Index)</li>
<li><b>Licence :</b> AGPL(Drivers : Apache)</li>
<li><b>Protocol :</b> BSON (Binary JSON)</li>
<li>Replication : Master/Slave Replication and automatic failover via Replica Sets</li>
<li>Sharding : Built-in</li>
<li>Queries are javascript expressions.</li>
<li>Runs arbitary javascript function server side.</li>
<li>Better Update-in-place than CouchDb.</li>
<li>Uses memory mapped files for data storage.</li>
<li>Performance over features.</li>
<li>Journaling (with --journal ) option turned on starting th mongod server.</li>
<li>Has Geospatial Indexing.</li>
<li>On 32-bit systems limited to 2.5GB.</li>
<li><span style="background-color: #f4f4f4; border: 0px; font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif; font-size: 15px; line-height: 22px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><b>Best used</b></span><strong style="background-color: #f4f4f4; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px; outline: 0px; padding: 0px; text-align: -webkit-auto; vertical-align: baseline;">:</strong><span style="background-color: #f4f4f4; font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif; font-size: 15px; line-height: 22px;"> If you need dynamic queries. If you prefer to define indexes, not map/reduce functions. If you need good performance on a big DB. If you wanted CouchDB, but your data changes too much, filling up disks.</span></li>
<li><strong style="background-color: #f4f4f4; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px; outline: 0px; padding: 0px; text-align: -webkit-auto; vertical-align: baseline;">For example:</strong><span style="background-color: #f4f4f4; font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif; font-size: 15px; line-height: 22px;"> For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back.</span></li>
</ul>
<div>
<br /></div>
<span style="color: #351c75;">Cassandra</span><br />
<div>
<br />
<div>
<ul style="background-color: #f4f4f4; border: 0px; list-style-image: initial; list-style-position: initial; margin: 0px 0px 20px; outline: 0px; padding: 0px; text-align: left; vertical-align: baseline;">
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Written in:</strong> Java</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Main point:</strong> Best of BigTable and Dynamo</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">License:</strong> Apache</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Protocol:</strong> Custom, binary (Thrift)</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Tunable trade-offs for distribution and replication (N, R, W)</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Querying by column, range of keys</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">BigTable-like features: columns, column families</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Has secondary indices</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Writes are much faster than reads (!)</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Map/reduce possible with Apache Hadoop</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">All nodes are similar, as opposed to Hadoop/HBase</li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; text-align: -webkit-auto; vertical-align: baseline;">Best used:</strong><span style="text-align: -webkit-auto;"> </span><span style="text-align: -webkit-auto;">When you write more than you read (logging). If every component of the system must be in Java. ("No one gets fired for choosing Apache's stuff.")</span></li>
<li style="background-color: transparent; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; text-align: -webkit-auto; vertical-align: baseline;">For example:</strong><span style="text-align: -webkit-auto;"> </span><span style="text-align: -webkit-auto;">Banking, financial industry (though not necessarily for financial transactions, but these industries are much bigger than that.) Writes are faster than reads, so one natural niche is real time data analysis.</span><span style="background-color: transparent;"> </span></li>
</ul>
<div>
<br />
<span style="color: #351c75;">HBase</span><br />
<br />
<br />
<ul style="background-color: #f4f4f4; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; list-style-image: initial; list-style-position: initial; margin: 0px 0px 20px; outline: 0px; padding: 0px; text-align: -webkit-auto; vertical-align: baseline;">
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Written in:</strong> Java</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Main point:</strong> Billions of rows X millions of columns</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">License:</strong> Apache</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Protocol:</strong> HTTP/REST (also Thrift)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Modeled after Google's BigTable</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Uses Hadoop's HDFS as storage</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Map/reduce with Hadoop</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Query predicate push down via server side scan and get filters</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Optimizations for real time queries</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">A high performance Thrift gateway</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">HTTP supports XML, Protobuf, and binary</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Cascading, hive, and pig source and sink modules</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Jruby-based (JIRB) shell</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Rolling restart for configuration changes and minor upgrades</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Random access performance is like MySQL</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">A cluster consists of several different types of nodes</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Best used:</strong> Hadoop is probably still the best way to run Map/Reduce jobs on huge datasets. Best if you use the Hadoop/HDFS stack already.</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">For example:</strong> Analysing log data.</li>
</ul>
<br />
<span style="color: #351c75;">CouchDB</span><br />
<br />
<br />
<ul style="background-color: #f4f4f4; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; list-style-image: initial; list-style-position: initial; margin: 0px 0px 20px; outline: 0px; padding: 0px; text-align: -webkit-auto; vertical-align: baseline;">
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Written in:</strong> Erlang</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Main point:</strong> DB consistency, ease of use</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">License:</strong> Apache</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Protocol:</strong> HTTP/REST</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Bi-directional (!) replication,</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">continuous or ad-hoc,</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">with conflict detection,</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">thus, master-master replication. (!)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">MVCC - write operations do not block reads</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Previous versions of documents are available</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Crash-only (reliable) design</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Needs compacting from time to time</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Views: embedded map/reduce</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Formatting views: lists & shows</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Server-side document validation possible</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Authentication possible</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Real-time updates via _changes (!)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Attachment handling</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">thus, <a href="http://couchapp.org/" style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">CouchApps</a> (standalone js apps)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">jQuery library included</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Best used:</strong> For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important.</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">For example:</strong> CRM, CMS systems. Master-master replication is an especially interesting feature, allowing easy multi-site deployments.</li>
</ul>
<div style="text-align: -webkit-auto;">
<span style="color: #351c75; font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif;"><span style="font-size: 15px; line-height: 22px;">Neo4j</span></span></div>
<div style="text-align: -webkit-auto;">
<span style="font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif;"><span style="font-size: 15px; line-height: 22px;"><br /></span></span></div>
<div style="text-align: -webkit-auto;">
<ul style="background-color: #f4f4f4; border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 15px; line-height: 22px; list-style-image: initial; list-style-position: initial; margin: 0px 0px 20px; outline: 0px; padding: 0px; vertical-align: baseline;">
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Written in:</strong> Java</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Main point:</strong> Graph database - connected data</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">License:</strong> GPL, some features AGPL/commercial</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Protocol:</strong> HTTP/REST (or embedding in Java)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Standalone, or embeddable into Java applications</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Full ACID conformity (including durable data)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Both nodes and relationships can have metadata</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Integrated pattern-matching-based query language ("Cypher")</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Also the "Gremlin" graph traversal language can be used</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Indexing of nodes and relationships</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Nice self-contained web admin</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Advanced path-finding with multiple algorithms</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Indexing of keys and relationships</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Optimized for reads</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Has transactions (in the Java API)</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Scriptable in Groovy</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;">Online backup, advanced monitoring and High Availability is AGPL/commercial licensed</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Best used:</strong> For graph-style, rich or complex, interconnected data. Neo4j is quite different from the others in this sense.</li>
<li style="background-color: transparent; border: 0px; margin: 0px 0px 0px 30px; outline: 0px; padding: 0px; vertical-align: baseline;"><strong style="background-color: transparent; border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">For example:</strong> Social relations, public transport links, road maps, network topologies</li>
</ul>
</div>
<div style="text-align: -webkit-auto;">
<span style="font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif;"><span style="font-size: 15px; line-height: 22px;"><br /></span></span>
<span style="font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif;"><span style="font-size: 15px; line-height: 22px;">Reffered Sources : </span></span><a href="http://kkovacs.eu/" style="text-align: left;">kkovacs</a> , <a href="http://en.wikipedia.org/">wikipedia</a></div>
<div style="text-align: -webkit-auto;">
<span style="font-family: "helvetica neue" , "helvetica" , "arial" , "liberation sans" , "freesans" , sans-serif;"><span style="font-size: 15px; line-height: 22px;"><br /></span></span></div>
</div>
</div>
</div>
</div>
Vishnu Chilamakuruhttp://www.blogger.com/profile/13099618632034328290noreply@blogger.com0tag:blogger.com,1999:blog-5894846330341913176.post-77107799710980319052012-02-21T04:49:00.001-08:002016-02-29T07:02:33.785-08:00GraphDatabase - The future for Facebook Recommendations<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<b>On what Basis are you getting Recommendations from Facebook??How your data is stored Internally in Social Network Sites ??</b><br />
<div>
<b><br /></b></div>
<div>
Have you ever thought how your information is stored by facebook in database?? Do you think its SQL that facebook is using for storing your data ?? If you think so ,then you are wrong.Its NoSQL GraphDatabase called 'Cassandra' what facebook uses to store your data.I know after reading this you will get lot of questions in your mind. 'What is Graph database?? How it looks like?? How it can be useful for Facebook Recommendations?? Where else it can be used??'.Let me explain each one in detail.</div>
<div>
<br /></div>
<div>
<b>What is Graph database??</b></div>
<div>
<b><br /></b></div>
<div>
I think Wikipedia gives the best answer for this question.So i think i can just add a link to wikipedia for the introduction of graphDatabase. <a href="http://en.wikipedia.org/wiki/Graph_database">Here</a> you go..!!<br />
<br />
<b>How it looks like??</b><br />
<br />
I thing you got a basic idea about graph database after seeing Wikipedia page.Here i am showing sample example of a small Social Network of friends who KNOWS each other.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://29.media.tumblr.com/tumblr_lxj098nBYo1qz7jv1o1_500.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://29.media.tumblr.com/tumblr_lxj098nBYo1qz7jv1o1_500.png" height="291" width="320" /></a></div>
<br /></div>
<br />
<div>
You can Imagine the entire Facebook database as a infinite Graph where the users keep on increasing day by day.Some thing like this<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://t1.gstatic.com/images?q=tbn:ANd9GcTEKkdTzsApufU3NxraRtTmJ-QbDLfmnq-S95mdg68kZBVCTxHSjg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://t1.gstatic.com/images?q=tbn:ANd9GcTEKkdTzsApufU3NxraRtTmJ-QbDLfmnq-S95mdg68kZBVCTxHSjg" /></a></div>
Where each node represents each Facebook user or page and each edge between two users represents a FRIEND and LIKE relationship.<br />
<b><br /></b></div>
<div>
<div style="font-weight: bold;">
<b>How it can be useful for Facebook Recommendations??</b></div>
<div style="font-weight: bold;">
<b><br /></b></div>
Consider the sample example of a small graph which has 3 users A,B,C.<br />
<br />
<ol style="text-align: left;">
<li>A - friend of B </li>
<li>B - friend of A,C </li>
<li>C - friend of B</li>
</ol>
Now if u notice Facebook recommends 'C to A' and 'A to C ' to make friendship each other as 'user B' is the common friend between them.And this is as simple as to find the common node between two edges in GraphDatabase.<br />
<br />
If you use SQL u need to join all 3 records together based on 'friends' field and need to find out the transitive relationship between A,B,C which is time taking.<br />
<br />
The above example is a very basic one.More recommendations can be found out using mutual LIKES between two users,games,pages,etc...what not..!! These things can be easily implemented using GraphDatabase and it is very efficient than SQL. <br />
<div style="font-weight: bold;">
<b><br /></b></div>
<div style="font-weight: bold;">
<b>Where else it can be used??</b></div>
<div style="font-weight: bold;">
<b><br /></b></div>
I feel GraphDatabases are very efficient to use for social networking,spatial search,recommendation engines(Ex: Amazon,Facebook),etc ....<br />
<div style="font-weight: bold;">
<br /></div>
</div>
</div>
<b>
</b>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-29372173-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script></div>
Vishnu Chilamakuruhttp://www.blogger.com/profile/13099618632034328290noreply@blogger.com0tag:blogger.com,1999:blog-5894846330341913176.post-26644459572815502482012-02-21T02:38:00.000-08:002016-02-29T07:02:51.125-08:00Why Nooooo SQL .........???<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<br />
Relational databases have been around for many decades and are the database technology of choice for most traditional data-intensive storage and retrieval applications. Retrievals are usually accomplished using SQL, a declarative
query language. Relational database systems are generally
efficient unless the data contains many relationships requiring joins of large tables. Recently there has been much interest in data stores that do not use SQL exclusively, the so called NoSQL movement. Examples are Google’s <a href="http://en.wikipedia.org/wiki/BigTable">BigTable</a>
and Facebook’s <a href="http://en.wikipedia.org/wiki/Cassandra">Cassandra</a>. Lets have a look at <a href="http://en.wikipedia.org/wiki/NoSQL">NoSQL</a> vs <a href="http://en.wikipedia.org/wiki/MySQL">MySQL</a> (common relational database system).<br />
<a href="http://www.geminimobile.com/images/subpages/logos-NoSQL.jpg" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" src="http://www.geminimobile.com/images/subpages/logos-NoSQL.jpg" height="105" width="200" /> </a><a href="http://www.geminimobile.com/images/subpages/logos-Cassandra.jpg" imageanchor="1" style="clear: right; display: inline !important; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://www.geminimobile.com/images/subpages/logos-Cassandra.jpg" height="168" width="320" /></a><br />
<br />
<b>When to go for NOSQL ??</b><br />
<br />
In recent years, software developers have been investigating storage alternatives to relational databases. NoSQL is a blanket term for some of those new systems. Cassandra,BigTable, CouchDB, Project Voldemort, and Dynamo are all NoSQL projects, as they are all high-volume data stores that actively reject the relational and object-relational models.<br />
<div style="font-weight: bold;">
<br /></div>
<div>
<div>
Atomicity, consistency, isolation, and durability (ACID) are a set of governing principles of the relational model. Together, they guarantee database reliability. NoSQL rejects ACID.</div>
<div>
<br /></div>
<div>
The term “NoSQL,” as a term for modern web data stores,first began to gain popularity in early 2009. It is a topic that has gained recognition from the IT community but has yet to garner large-scale academic study. Still, the NoSQL movement has its own discussion groups, blogs, and conferences.</div>
<div>
<br /></div>
<div>
As the typical database administrator attempts to question whether to move from the relational model to a NoSQL model, the NoSQL community presents him or her with potential flags that the data might be more suitable for a NoSQL system.</div>
<div>
<ol>
<li>Having tables with lots of columns, each of which is only used by a few rows.</li>
<li>Having attribute tables.</li>
<li>Having lots of many-to-many relationships.</li>
<li>Having tree-like characteristics.</li>
<li>Requiring frequent schema changes.</li>
</ol>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
</div>
</div>
<br /></div>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-29372173-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</div>
Vishnu Chilamakuruhttp://www.blogger.com/profile/13099618632034328290noreply@blogger.com0