By Abhilash | October 9, 2019
Recently I worked on refactoring a piece of code which dealt with filtering DB entities based on certain attributes and there were different types of such filters each doing almost the same thing and the problem was, how to reuse the common logic while making sure that the specific details of filtering are the only thing each filter will have to worry about.
To better explain the problem and its solution, consider an example of a banking system, different entities in the system are the client (the account holder), account, deposit, withdrawal, investment, investment sale, etc.
A deposit or a withdrawal has a reference to an account which in turn has a client reference, since an account belongs to a client, and we can perform deposit/ withdrawal transactions on it, we can say, “get all deposits for a client and/or account”(a client can have more than one account), similarly withdrawals also can be filtered by a client and/or account.
An investment sale has a reference to an investment which in turn has an account reference since we typically invest a certain amount from an account on an asset (like stock) and later we can liquidate or sell the investment as well. To filter & retrieve investment sales, we can say “get a list of investment sales for a certain client and/or account and/or investment”.
To achieve the above filtering we have implemented two different filter managers here which are ClientOrAccountFilterManager and ClientOrAccountOrInvestmentFilterManager. Each of them intends to filter entities that implement the interface IHasAccountReference and IHasInvestmentReference respectively.
Entities to be filtered
public class DbDeposit : IHasAccountReference
{
public DbAccount Account {get; set;} //this is from account reference
// Other members follow ...
}
public class DbInvestmentSale : IHasInvestmentReference
{
public DbInvestment Investment {get; set;} //this is from investment reference
// Other members follow ...
}
Filter criteria
public interface IClientOrAccountFilterCriteria
{
Guid AccountId {get; set;}
string AccountNumber {get; set;}
Guid ClientId {get; set;}
string ClientNumber {get; set;}
}
public interface IClientOrAccountOrInvestmentFilterCriteria
{
Guid AccountId {get; set;}
string AccountNumber {get; set;}
Guid ClientId {get; set;}
string ClientNumber {get; set;}
Guid InvestmentId {get; set;}
}
Current filter implementation
To keep things simple and to the point we’ll use pseudo code where necessary to outline the method that filters by client and/or account in ClientOrAccountFilterManager.
public Task<List<TModelType>> FilterByClientOrAccountAsync<TModelType>(
IClientOrAccountFilterCriteria filterCriteria,
PagingOptions pagingOptions,
Func<IQueryable<TDbType>, IQueryable<TDbType>> filter = null) where TDbType : IHasAccountReference
{
//1. Check if filterCriteria is null, if yes throw an argument null exception
//2. Check if at least one of the filterCriteria are provided, if not then throw insufficient filters exception
//3. Filter by client and/or account
// for instance one of the predicates would be TDbType.Account.AccountId == filterCriteria.AccountId
// or TDbType.Account.Client.ClientId == filterCriteria.ClientId
//4. Apply additional filters supplied
if (filter != null)
{
filterQuery = filter(filterQuery);
}
//5. Step no 3 & 4 return an IQueryable<TDbType> apply pagination logic on it using the pagingOptions supplied
// for instance getting a certain page of data by pageNumber
//6. If no items were found after applying above filters, check if the filterCriteria supplied are valid
// identifiers, such as check if filterCriteria.AccountId if supplied, is a valid id in the database, if not throw
// an exception
//7. If we are here then map the List<TDbType> to the List<TModelType> and return
}
Similarly, the method that filters by client/account/investment in ClientOrAccountOrInvestmentFilterManager can be expressed as below.
public Task<List<TModelType>> FilterByClientOrAccountOrInvestmentAsync<TModelType>(
IClientOrAccountOrInvestmentFilterCriteria filterCriteria,
PagingOptions pagingOptions,
Func<IQueryable<TDbType>, IQueryable<TDbType>> filter = null) where TDbType : IHasInvestmentReference
{
//1. Check if filterCriteria is null, if yes throw an argument null exception
//2. Check if at least one of the filterCriteria are provided, if not then throw insufficient filters exception
//3. Filter by client and/or account and/or investment
// for instance one of the predicates would be TDbType.Investment.Account.AccountId == filterCriteria.AccountId
// or TDbType.Investment.Account.Client.ClientId == filterCriteria.ClientId
// or TDbType.InvestmentId == filterCriteria.InvestmentId
//4. Apply additional filters supplied
if (filter != null)
{
filterQuery = filter(filterQuery);
}
//5. Step no 3 & 4 return an IQueryable<TDbType> apply pagination logic on it using the pagingOptions supplied
// for instance getting a certain page of data by pageNumber
//6. If no items were found after applying above filters, check if the filterCriteria supplied are valid
// identifiers, such as check if filterCriteria.AccountId if supplied, is a valid id in the database, if not throw
// an exception
//7. If we are here then map the List<TDbType> to the List<TModelType> and return
}
Template method pattern
As you can see the above filter methods follow the same steps to arrive at a filtered result. The only different thing is the filter criteria and the TDbType constraint, the first filter method works on an IHasAccountReference TDbType and the second one works on TDbType of IHasInvestmentReference.
One of the reasons why the above methods cannot be combined into a single method is provided in this SO answer, in short, we cannot have
public Task<List<TModelType>> FilterByClientOrAccountOrInvestmentAsync<TModelType>(
IFilterCriteria filterCriteria,
PagingOptions pagingOptions,
Func<IQueryable<TDbType>, IQueryable<TDbType>> filter = null) where TDbType : IHasAccountReference, IHasInvestmentReference
Since the methods cannot be combined we’ll have to see if the process they follow can be generalized and then leave the specifics to be implemented by each filter manager, incidentally, this is exactly what the template method pattern suggests, quoting from the wiki article
In object-oriented programming, the template method is one of the behavioral design patterns identified by Gamma et al. in the book Design Patterns. The template method is a method in a superclass, usually an abstract superclass, and defines the skeleton of an operation in terms of a number of high-level steps. These steps are themselves implemented by additional helper methods in the same class as the template method.
The helper methods may be either abstract methods, for which case subclasses are required to provide concrete implementations, or hook methods, which have empty bodies in the superclass. Subclasses can (but are not required to) customize the operation by overriding the hook methods. The intent of the template method is to define the overall structure of the operation, while allowing subclasses to refine, or redefine, certain steps.
Applying the pattern in FilteringManagerBase
Based on this idea we can create an abstract base as below
public abstract class FilteringManagerBase<TDbType, TFilterCriteria>
where TFilterCriteria : IFilterCriteria
{
public async Task<List<TModelType>> FilterAsync<TModelType>(
TFilterCriteria filterCriteria,
PagingOptions pagingOptions,
Func<IQueryable<TDbType>, IQueryable<TDbType>> filter = null)
{
//1. Check if filterCriteria is null, if yes throw an argument null exception
if (filterCriteria is null)
{
throw new InvalidDomainOperationException($"{nameof(filterCriteria)} cannot be null", ErrorCodes.FilterArgumentNull);
}
//2. Check if at least one of the filterCriteria are provided, if not then throw insufficient filters exception
if (filterCriteria.IsEmpty())
{
throw new InvalidDomainOperationException("Insufficient filters specified", ErrorCodes.InsufficientFiltersSpecified);
}
//3. Filter by the right filter criteria which, is detailed by the sub class implementation
var filterQuery = await GetFilterQuery(filterCriteria).ConfigureAwait(false);
if (filter != null)
{
filterQuery = filter(filterQuery);
}
var dbTypesQuery = filterQuery.Paginate(filterCriteria, out var totalCount, ref pagingOptions, this.mapper).ToListAsync().ConfigureAwait(false);
// If no items were found, check whether the filtering entity exists and if not return appropriate error code
if (!dbTypes.Any())
{
await AssertEntityExists(filterCriteria).ConfigureAwait(false);
}
return this.mapper.Map<List<TModelType>>(dbTypes);
}
protected abstract Task AssertEntityExists(TFilterCriteria filterCriteria);
protected abstract Task<IQueryable<TDbType>> GetFilterQuery(TFilterCriteria filterCriteria);
}
public interface IFilterCriteria
{
bool IsEmpty();
}
As you can decipher from the above code, it follows the definition for the template method pattern.
FilterAsync provides the overall structure of the operation while the helper methods AssertEntityExists, FilterParametersEmpty, GetFilterQuery are overridden by the specific filters which then completes the filter operation definition.
Overriding specifics in subclasses
So the subclasses now inherit the FilteringManagerBase abstract class and follow the structure provided by FilterAsync but define their filtering behavior by overriding the helper methods AssertEntityExists, FilterParametersEmpty, GetFilterQuery as shown below.
public sealed class ClientOrAccountFilteringManager<TDbType, TQuery, TFilterCriteria> : FilteringManagerBase<TDbType, TFilterCriteria>, IClientOrAccountFilteringManager<TDbType, TFilterCriteria>
where TDbType : IHasAccountReference
where TQuery : IQuery<TDbType, Guid>
where TFilterCriteria : IFilterCriteria, , IClientOrAccountFilterCriteria
{
protected override async Task AssertEntityExists(TFilterCriteria clientOrAccountFilterCriteria)
{
// If any client details are passed in, check that the client actually exists
if (clientOrAccountFilterCriteria.AreClientFiltersProvided())
{
var dbClient = await this.clientQuery.GetByReferenceAsync(
clientOrAccountFilterCriteria.ClientId,
clientOrAccountFilterCriteria.ClientNumber).ConfigureAwait(false);
if (dbClient == null)
{
throw new InvalidDomainOperationException("Client not found", ErrorCodes.ClientNotFound);
}
}
// If any account details are passed in, check that the account actually exists for the client (in the case that client details were also passed in)
if (clientOrAccountFilterCriteria.AreAccountFiltersProvided())
{
var dbAccount = await this.accountQuery.GetByAnyReferenceAsync(
clientOrAccountFilterCriteria.AccountId,
clientOrAccountFilterCriteria.AccountNumber,
clientOrAccountFilterCriteria.ClientId,
clientOrAccountFilterCriteria.ClientNumber).ConfigureAwait(false);
if (dbAccount == null)
{
throw new InvalidDomainOperationException("Account not found", ErrorCodes.AccountNotFound);
}
}
}
protected override async Task<IQueryable<TDbType>> GetFilterQuery(TFilterCriteria clientOrAccountFilterCriteria)
{
return (await this.query.GetAllAsync().ConfigureAwait(false))
.FilterByIdAndNumber(
x => x.Account.Client.Id,
x => x.Account.Client.ClientNumber,
clientOrAccountFilterCriteria.ClientId,
clientOrAccountFilterCriteria.ClientNumber,
ErrorCodes.ClientFilterNotSpecified
)
.FilterByIdAndNumber(
x => x.Account.Id,
x => x.Account.AccountNumber,
clientOrAccountFilterCriteria.AccountId,
clientOrAccountFilterCriteria.AccountNumber,
ErrorCodes.AccountFilterNotSpecified
);
}
}
}
public class ClientOrAccountOrInvestmentFilteringManager<TDbType, TQuery, TFilterCriteria> : FilteringManagerBase<TDbType, TFilterCriteria>, IClientOrAccountOrInvestmentFilteringManager<TDbType, TFilterCriteria>
where TDbType : IHasInvestmentReference
where TQuery : IQuery<TDbType, Guid>
where TFilterCriteria : IFilterCriteria, IClientOrAccountOrInvestmentFilterCriteria
{
protected override async Task AssertEntityExists(TFilterCriteria clientOrAccountOrInvestmentFilterCriteria)
{
// If any client details are passed in, check that the client actually exists
if (clientOrAccountOrInvestmentFilterCriteria.AreClientFiltersProvided())
{
var dbClient = await this.clientQuery.GetByReferenceAsync(
clientOrAccountOrInvestmentFilterCriteria.ClientId,
clientOrAccountOrInvestmentFilterCriteria.ClientNumber).ConfigureAwait(false);
if (dbClient == null)
{
throw new InvalidDomainOperationException("Client not found", ErrorCodes.ClientNotFound);
}
}
// If any account details are passed in, check that the account actually exists for the client (in the case that client details were also passed in)
if (clientOrAccountOrInvestmentFilterCriteria.AreAccountFiltersProvided())
{
var dbAccount = await this.accountQuery.GetByAnyReferenceAsync(
clientOrAccountOrInvestmentFilterCriteria.AccountId,
clientOrAccountOrInvestmentFilterCriteria.AccountNumber,
clientOrAccountOrInvestmentFilterCriteria.ClientId,
clientOrAccountOrInvestmentFilterCriteria.ClientNumber).ConfigureAwait(false);
if (dbAccount == null)
{
throw new InvalidDomainOperationException("Account not found", ErrorCodes.AccountNotFound);
}
}
// If any investment details are passed in, check that the investment actually exists for the account (in the case that account and client details were also passed in)
if (clientOrAccountOrInvestmentFilterCriteria.AreInvestmentFiltersProvided())
{
var dbInvestment = await this.investmentQuery.GetByAnyReferenceAsync(
clientOrAccountOrInvestmentFilterCriteria.InvestmentId,
clientOrAccountOrInvestmentFilterCriteria.AccountId,
clientOrAccountOrInvestmentFilterCriteria.AccountNumber,
clientOrAccountOrInvestmentFilterCriteria.ClientId,
clientOrAccountOrInvestmentFilterCriteria.ClientNumber).ConfigureAwait(false);
if (dbInvestment == null)
{
throw new InvalidDomainOperationException("Investment not found", ErrorCodes.InvestmentNotFound);
}
}
}
protected override async Task<IQueryable<TDbType>> GetFilterQuery(TFilterCriteria clientOrAccountFilterCriteria)
{
return (await this.query.GetAllAsync().ConfigureAwait(false))
.FilterByIdAndNumber(
x => x.Account.Client.Id,
x => x.Account.Client.ClientNumber,
clientOrAccountFilterCriteria.ClientId,
clientOrAccountFilterCriteria.ClientNumber,
ErrorCodes.ClientFilterNotSpecified
)
.FilterByIdAndNumber(
x => x.Account.Id,
x => x.Account.AccountNumber,
clientOrAccountFilterCriteria.AccountId,
clientOrAccountFilterCriteria.AccountNumber,
ErrorCodes.AccountFilterNotSpecified
)
.FilterByInvestmentFilter(clientOrAccountOrInvestmentFilter, ErrorCodes.InvestmentFilterNotSpecified);
}
}