category:Uncategorized

esProc Implements Foreign Key Relationship for MongoDB Collections

Problem source:http://stackoverflow.com/questions/29392169/populating-field-values-for-referred-documents-in-aggregate-call-in-mongoose . With MongoDB’s built-in API, you implement a foreign key relationship through hardcoding. The hardcode is not intuitive and difficult to write. In this case you can use esProc to han...

2015-04-10 1061 0 0

esProc Exports Data by Groups

Problem source:http://bbs.csdn.net/topics/390500337 . A selection from the original table (TB1): Note: The table has a great amount of data volume with 4 million rows of data. You need to export data into several TXTfiles according to different values in PROGROUP column. Based on the above data, the expected resu...

2015-04-09 1018 0 0

Query List Fields in MongoDB Subdocuments in esProc

Problem source:https://groups.google.com/forum/#!msg/mongodb-user/HqzXSh5DZek/ffZG0TQ1w8cJ. Collection Cbetween contains cascaded subdocuments, in which the List-type dataList field includes a series of strings, each of which has multiple numbers. You need to find strings according to the criterion that the first numbe...

2015-04-08 1144 0 0

esProc Exports Unstructured MongoDB Data as CSV Files

Problem source:https://plus.google.com/+VicNgrail/posts/ebS9JUtFopw . MongoDB allows storing unstructured data in it. But it is somewhat difficult to export the data as standard structured data. esProc, however, makes it an easy job, with MongoDB’s cooperation. Let’s look at the steps for doing this. Below is some data...

2015-04-07 961 0 0

Comparison between esProc Syntax and SQL Syntax for Common Computations

Structured Query Language, abbreviated as SQL, is usually used to query, store and retrieve data in databases. In esProc, users can return the query result using SQL commands directly, or process data step by step with more flexible and convenient methods and get the same result as that of SQL, whose computation is com...

2015-04-03 954 0 0

esProc Performs Inner and Outer Joins: Code Examples

Related computing includes inner join and outer join. The outer join can be further divided into left join, right join, and full join. esProc performs these joins conveniently. In the discussion below, we’ll learn how esProc does the job using the emp table and the sOrder table along with several examples. Table struct...

2015-04-01 993 0 0

esProc Joins a Structured Text file with a JSON File

structure.txt is a tab-separated structured text file. json.txt contains non-structured JSON strings. We need to join the two files to create a new file result.txt. The original data is as follows: structure.txt Json.txt Result.txt esProc will first import json.txt as a structured table sequence and then join it wit...

2015-03-31 895 0 0

esProc Joins Text files and Generates a Computed Column

There are two tab-separated structured text files. chr column in AssociatedMarkers.txt is the logical foreign key pointing to Chr column in DiseaseMarkers.txt. We want to create a new structured text file, in which one column comes from AssociatedMarkers.txt’s snps_BCG24 column and the other is a computed column that w...

2015-03-30 917 0 0

An Example of esProc Set Operations

There are a lot of occasions when set operations are needed to do structured data summarizing and analysis. For example, to list all students who have published papers, find the employees who have participated in all trainings and select students who passed the examination for the re-examination. esProc uses sets every...

2015-03-27 908 0 0

esProc Finds Differences between CSV files

userName and date are the logical primary key of both old.csv file and new.csv file, in which we want to find rows that are new, deleted and updated. The source data is as follows: As can be seen from the above data, in new.csv the 2nd and the 3rd row are the new and the 4th row is the updated; in old.csv the 3rd row ...

2015-03-26 1036 0 0

esProc Counts Distinct Columns in a Text File

Problem source:http://unix.stackexchange.com/questions/161885/using-awk-to-identify-the-number-identical-columns . There are some text files under /data directory. Each of them has certain columns. We want to know how many distinct columns are there in each file. For instance, the number of distinct columns in f1.txt i...

2015-03-25 997 0 0

esProc Performs Random UPDATE with Priorities

Problem Source: https://www.linkedin.com/groups/update-records-based-on-rank-1890173.S.5914444354742661123?trk=groups_items_see_more-0-b-ttl . Target:Among the records that satisfy the specified condition in the TMP_SURVEY_TRAN_BZ_3_WORKING table, select 20 ones at random and change values of their Quota_Include_Ind fi...

2015-03-24 1012 0 0

esProc Assists Java to Join Text Files

Problem Source: http://stackoverflow.com/questions/26820118/text-file-processing-using-java . JAVA lacks the class library for handling text file JOIN operation. So it’s quite complicated to write code specifically for it, especially when the files are too big to be loaded into the memory all at once. esProc can help J...

2015-03-23 1032 0 0

esProc Simplifies SQL-style Computations – Interval Merging

There are many complicated SQL-style computations in database application development. Interval merging is one of them, such as the summation of non-overlapping time intervals and merging of overlapping time intervals, etc. Due to the lack of orderliness of an SQL set, SQL solves the problems by resorting to recursive ...

2015-03-20 971 0 0

An Illustration of esProc’s Parallel Processing of Big Text Files

In esProc we can perform parallel processing on big text files conveniently. The following example will illustrate the method.  There is a text file, sales.txt, with ten million sales records. Its main fields include SellerID, OrderDate and Amount. Now compute each salesman’s total amount of big orders in the past...

2015-03-19 837 0 0