category:Uncategorized

Examples of Handling Structured Text Files with esProc

Structured text files are a common file format, and the tasks of handling them are common too. One approach that comes to mind easily is to import such a file into the database for manipulation. But this is time-consuming and causes heavy database usage. In some cases, there is even not a database available. It’s conve...

2016-03-28 1826 0 0

Examples of Dynamic-Column-SQL Computations in JAVA

It’s tedious and cumbersome to express dynamic columns in SQL, so programmers usually turn to high-level languages like JAVA to compose the dynamic statement. The problem is JAVA basic class libraries don’t include set operations, causing equal amount of difficulty for those trying to do it automatically. esProc suppor...

2015-12-15 1246 0 0

How esProc Assists Writing SQL Queries

esProc can simplify complex SQL-style queries using ordered sets, object-style access and stepwise computation. Usually there are two ways in which esProc is used: an independent one and one requiring integration with Java. Now let’s look at the first one through an example.   The sales table stores several years ...

2015-07-30 1747 0 0

Performance Test of esProc Script Execution

Sometimes we may encounter computations too complicated to be handled solely using library functions. We need to hardcode the computational logic using scripts. In that case the interpretative and executional performance of the script becomes really important. Not a few traditional database tasks, being unable to be ex...

2015-05-26 1081 0 0

Importing Excel Data into Access with esProc

In daily work we have frequent use of text data or spreadsheets. We need to import data into database for further statistical analytics. For this task, esProc is a very handy tool. In the following example, we will import Excel data into an Access database, to demonstrate how to migrate text data into database with esP...

2015-05-25 3497 0 0

Group MongoDB Collection and Find Top N members in esProc

Problem source:https://groups.google.com/forum/#!topic/mongodb-user/Rjv6lyUCe1s . Collection last3 has two fields: variable and timestamp. You need to first group documents by variable and find from each group the top 3 ones with the latest timestamp, and then find from the three documents the one with the earliest tim...

2015-05-22 1246 0 0

esProc Assists Java in Merging Text Files to Excel

Problem source:http://bbs.csdn.net/topics/390842850 .  In this case, you need to merge data from all text files in a given directory and export it to an Excel file. Each of these text files has tab-seperated data with one row and three columns.  JAVA approaches the problem by running a loop to retrieve a...

2015-05-21 1198 0 0

Find Differences between Text Files with esProc

Problem source:http://bbs.csdn.net/topics/360033579 .  Compare the following two text files of the same format and write the differences (different records) into a text file. Content of 1.txt: Proto      Local Address Foreign Address     State TCP 111.11.1.111:1975...

2015-05-20 1177 0 0

Combine Text Files Conditionally with esProc

Problem source: http://bbs.csdn.net/topics/390507610 . There are multiple text files in a single directory which need to be combined according to specified conditions. The text files include, for example, 12345.txt, 12346.txt, 12347.txt, 2013070312345.txt, 2013070312346.txt, 2013070312347.txt and 2013070412347.txt...

2015-05-18 1092 0 0

Export Dynamic Text Files to Database with esProc

Problem source: http://bbs.csdn.net/topics/330190185 .  There are multiple text files in a certain directory. Each of their names includes one of the letters a/b/c at the beginning and a date. The letter is fixed while the date is dynamic. You need to write contents of these files as well as the dates into a ...

2015-05-15 1102 0 0

esProc Improves Text Processing – Remove Duplicate Rows

During processing the text file, sometimes we need to remove duplicate rows from the grouped data. But the operation becomes complicated when the file under processing is too big to be entirely loaded into the memory. esProc’s group operation supports importing a whole group of data with the file cursor, as well as man...

2015-05-14 1111 0 0

esProc Improves Text-processing – Merge Sorted Data

Many databases support the MERGE statement and thus can merge two tables conveniently. Database syntax cannot be used directly, however, if the data to be merged are the text data. In this case esProc is able to perform what MERGE statements can do based on the text files.    The two files – table1.txt ...

2015-05-13 920 0 0

esProc File Computing: Parallel Grouping and Aggregation

The article esProc File Computing: Parallel Query and Filter explained how to query and filter data. Here let’s look at the case of combining the article’s topic with grouping and aggregation. Still we only discuss cases with relatively small result sets that can be loaded into memory in one go.   Steps of queryin...

2015-05-12 913 0 0

Processing Big Text Files with esProc Cursor: Code Examples

esProc supports cursor objects and provides related functions to process big text files conveniently. Below is an example. Suppose a text file, sales.txt, stores ten million sales records. Its fields mainly include SellerID, OrderDate and Amount. Requirement: Computing each seller’s total amount of big orders in the pa...

2015-05-11 41102 0 0

esProc Handles Duplicated Records in CSV files

Problem source:http://stackoverflow.com/questions/28976554/removing-duplicates-from-csv-and-rearranging-the-data-into-columns-with-java .  JAVA doesn’t have the class library for grouping data from a text file or getting distinct values, which results rather complicated coding. In dealing with the duplicated ...

2015-05-07 1030 0 0