Data Sources

DataSourceComponents represent a source of tuples in Squall. This page briefly describes how to create these components and how to define new types of sources.

Creating a DataSourceComponent

There is two different ways of creating a DataSourceComponent in Squall: by directly creating an instance, or through a SquallContext object.

Directly creating an instance

When calling the constructor with the table name and a configuration object, Squall will look for look for a file called "<tablename>.<DIP_EXTENSION>" in the DIP_DATA_PATH folder. You can find an example of this here.

When calling the constructor with the component name and a path, Squall will read the data directly from the given path. Here is an example of this usage.

Through SquallContext

An alternative and more powerful way of creating a DataSourceComponent is through the createDataSource method in the SquallContext object.

When given a table name, this method will search the context for an appropriate ReaderProvider. By default, this means to search for a file with the same name as the table ending with DIP_EXTENSION in the current directory or in several directories used in Squall for testing.

You can register new ReaderProviders in the context by invoking the registerReaderProvider method. For instance, to add a specific path to be searched:

  context.registerReaderProvider(new FileReaderProvider("/path/to/tables/"));

When looking for an appropriate provider, Squall will search in reverse with respect to the order in which the providers were registered (i.e. it will first try the provider that was registered last).

Defining new data sources

A data source can be anything that produces tuples. To create a new type of data source, it is necessary to define a new class implementing CustomReader, and a new class inheriting from ReaderProvider.

It is very easy to define a new CustomReader, as the only two methods that need to be implemented are close and readLine. The readLine method should return a string representing a tuple, in which tuple fields are separated by DIP_READ_SPLIT_DELIMITER (by default this is the character |).

In order to have DataSourceComponents that use a new CustomReader, it is necessary to define a ReaderProvider. Classes inheriting from ReaderProvider have to implement the method canProvide, which is used by the context to choose wich ReaderProvider can be used to provide a reader for a given table name, and the method getReaderForName, which returns a CustomReader for a given table name.

As an example, here is the definition of FileReaderProvider in Squall.

Remember that you need to register the ReaderProvider in the context as described above.