bookmarks (hide)574
- Pattern is a web mining module for the Python programming language. It bundles tools for data retrieval (Google + Twitter + Wikipedia API, web spider, HTML DOM parser), text analysis (rule-based shallow parser, WordNet interface, syntactical + semantical n-gram search algorithm, tf-idf + cosine similarity + LSA metrics), clustering and classification (k-means, KNN, SVM), and data visualization (graph networks).
- Atom Interface is a novel interactive visualization of single/multiple tree structures. It is based on the metaphor of electrons, atoms and molecules. For mo...
- DB2 Graph Store is an optimized way to store graph triples inside DB2 database. Support for the SPARQL query language Support for popular RDF Java APIs like JENA Support for HTTP SPARQL end-point via JOSEKI
- Updating an index of the web as documents are crawled requires continuously transforming a large repository of existing documents as new documents arrive. This task is one example of a class of data processing tasks that transform a large repository of data via small, independent mutations. These tasks lie in a gap between the capabilities of existing infrastructure. Databases do not meet the storage or throughput requirements of these tasks: Google's indexing system stores tens of petabytes of data and processes billions of updates per day on thousands of machines. MapReduce and other batch-processing systems cannot process small updates individually as they rely on creating large batches for efficiency. We have built Percolator, a system for incrementally processing updates to a large data set, and deployed it to create the Google web search index. By replacing a batch-based indexing system with an indexing system based on incremental processing using Percolator, we process the same number of documents per day, while reducing the average age of documents in Google search results by 50%.
- Giraph builds upon the graph-oriented nature of Pregel but additionally adds fault-tolerance to the coordinator process with the use of ZooKeeper as its centralized coordination service. Giraph follows the bulk-synchronous parallel model relative to graphs where vertices can send messages to other vertices during a given superstep. Checkpoints are initiated by the Giraph infrastructure at user-defined intervals and are used for automatic application restarts when any worker in the application fails. Any worker in the application can act as the application coordinator and one will automatically take over if the current application coordinator fails.
- SparQLed is an interactive SPARQL editor that provides context-aware recommendations, helping users in formulating complex SPARQL queries across multiple heterogeneous data sources.
- An simple analytics engine for cassandra. For big data, individual data-point may be less interesting that simple aggregates.
- Stream summarizer and cardinality estimator. Contribute to stream-lib development by creating an account on GitHub.
- Tulip is an information visualization framework dedicated to the analysis and visualization of relational data. Tulip aims to provide the developer with a complete library, supporting the design of interactive information visualization applications for relational data that can be tailored to the problems he or she is addressing.
- EnSight Post-processing and Visualization for Scientific Data
- Developed by the Lawrence Livermore National Laboratory, VisIt contains a rich set of visualization methods—such as contour plots, pseudocolor plots, volume plots, vector plots, and boundary plots—for visualizing scientific data. VisIt allows the ability to provide quantitative as well as qualitative information from a scientific data set.
- The Visualization ToolKit (VTK) is an open source, freely available software system for 3D computer graphics, image processing, and visualization used by thousands of researchers and developers around the world.
- ParaView is an open-source, multi-platform application designed to visualize data sets of size varying from small to very large.
- New Dutch cloud service Silk, which is launching today, wants to fulfill the promise of the Semantic Web and make your documents, web pages and files more powerful -- and with a few fixes, it could get there.
- Another cloud computing service for prediction. This time based on a massive joint-prob model across all variables.
- BigML is a web-service for running distributed ML. 64gb free upload, API in python.
- a nice example of integrating data-sources without rewriting the data.
publications (hide)
- No matching posts.
