Structured text files are a common file format, and the tasks of handling them are common too. One approach that comes to mind easily is to import such a file into the database for manipulation. But this is time-consuming and causes heavy database usage. In some cases, there is even not a database available. It’s conve...
2016-03-28 1812 0
0
It’s tedious and cumbersome to express dynamic columns in SQL, so programmers usually turn to high-level languages like JAVA to compose the dynamic statement. The problem is JAVA basic class libraries don’t include set operations, causing equal amount of difficulty for those trying to do it automatically.
esProc suppor...
2015-12-15 1238 0
0
esProc can simplify complex SQL-style queries using ordered sets, object-style access and stepwise computation. Usually there are two ways in which esProc is used: an independent one and one requiring integration with Java. Now let’s look at the first one through an example.
The sales table stores several years ...
2015-07-30 1735 0
0
Sometimes we may encounter computations too complicated to be handled solely using library functions. We need to hardcode the computational logic using scripts. In that case the interpretative and executional performance of the script becomes really important. Not a few traditional database tasks, being unable to be ex...
2015-05-26 1068 0
0
In daily work we have frequent use of text data or spreadsheets. We need to import data into database for further statistical analytics. For this task, esProc is a very handy tool.
In the following example, we will import Excel data into an Access database, to demonstrate how to migrate text data into database with esP...
2015-05-25 3490 0
0
Problem source：https://groups.google.com/forum/#!topic/mongodb-user/Rjv6lyUCe1s .
Collection last3 has two fields: variable and timestamp. You need to first group documents by variable and find from each group the top 3 ones with the latest timestamp, and then find from the three documents the one with the earliest tim...
2015-05-22 1239 0
0
Problem source：http://bbs.csdn.net/topics/390842850 .
In this case, you need to merge data from all text files in a given directory and export it to an Excel file. Each of these text files has tab-seperated data with one row and three columns.
JAVA approaches the problem by running a loop to retrieve a...
2015-05-21 1186 0
0
Problem source：http://bbs.csdn.net/topics/360033579 .
Compare the following two text files of the same format and write the differences (different records) into a text file.
Content of 1.txt：
Proto Local Address Foreign Address State
TCP 111.11.1.111:1975...
2015-05-20 1162 0
0
Problem source: http://bbs.csdn.net/topics/390507610 .
There are multiple text files in a single directory which need to be combined according to specified conditions. The text files include, for example, 12345.txt, 12346.txt, 12347.txt, 2013070312345.txt, 2013070312346.txt, 2013070312347.txt and 2013070412347.txt...
2015-05-18 1081 0
0
Problem source: http://bbs.csdn.net/topics/330190185 .
There are multiple text files in a certain directory. Each of their names includes one of the letters a/b/c at the beginning and a date. The letter is fixed while the date is dynamic. You need to write contents of these files as well as the dates into a ...
2015-05-15 1090 0
0
During processing the text file, sometimes we need to remove duplicate rows from the grouped data. But the operation becomes complicated when the file under processing is too big to be entirely loaded into the memory. esProc’s group operation supports importing a whole group of data with the file cursor, as well as man...
2015-05-14 1103 0
0
Many databases support the MERGE statement and thus can merge two tables conveniently. Database syntax cannot be used directly, however, if the data to be merged are the text data. In this case esProc is able to perform what MERGE statements can do based on the text files.
The two files – table1.txt ...
2015-05-13 908 0
0
The article esProc File Computing: Parallel Query and Filter explained how to query and filter data. Here let’s look at the case of combining the article’s topic with grouping and aggregation. Still we only discuss cases with relatively small result sets that can be loaded into memory in one go.
Steps of queryin...
2015-05-12 905 0
0
esProc supports cursor objects and provides related functions to process big text files conveniently. Below is an example.
Suppose a text file, sales.txt, stores ten million sales records. Its fields mainly include SellerID, OrderDate and Amount. Requirement: Computing each seller’s total amount of big orders in the pa...
2015-05-11 41094 0
0
Problem source：http://stackoverflow.com/questions/28976554/removing-duplicates-from-csv-and-rearranging-the-data-into-columns-with-java .
JAVA doesn’t have the class library for grouping data from a text file or getting distinct values, which results rather complicated coding. In dealing with the duplicated ...
2015-05-07 1017 0
0
