Wednesday, August 19, 2015

BIRT rptlibrary using JNDI datasource

Today it is a judgement day for me as I have to face yet another issue in my BIRT viewer implementation. The problem was related to a wide-spread data sources in each and every report I have in my application. Now the challenge was to fix this blunder in more effective and efficient fashion.

The Problem:

Our project was using BIRT viewer to preview actual report before it gets exported to PDF or any other format. We have several number of reports which were using their own datasources. Difficulty was while deployment of solution on server. We have to change all datasources by modifying each .rptdesign file.

Solution:

Finally we have start thinking about the solution and arrived at a stepping stone to create an .rptlibrary file and add following properties under your
 <oda-data-source></oda-data-source>
tag.

<property name="odaDriverClass">{JDBC DRIVER NAME}</property>
            <property name="odaURL">{JDBC URL}</property>
            <property name="odaUser">{JDBC USER}</property>
            <encrypted-property name="odaPassword" encryptionID="base64">{JDBC PASSWORD}</encrypted-property>

This approach really did the trick and we finally took a sigh of relief from all difficulties we faced while deploying application.

After spending some lenient days, this BIRT deployment again started throttling us. This time it was more of a configuration issue. We had our datasource declared at several places and we had to bring everything at a single place. Finally we came up with JNDI approach and used JNDI data source declared in context.xml file of $TOMCAT_HOME/conf.context.xml in BIRT .rptlibrary file

This is how you would do it

$TOMCAT_HOME/conf/context.xml

<Resource auth="Container" driverClassName="{DIRVER NAME}" maxActive="20" maxIdle="2" maxWait="-1"
 name="/{DS NAME}" password="{DB PASSWORD}" removeAbandoned="true" removeAbandonedTimeout="20"
 type="javax.sql.DataSource" url="{JDBC URL}" username="{DB USER NAME}"
 validationQuery="select true;"/>

.rptlibrary file

add following line in your <oda-data-source></oda-data-source> tag

 <property name="odaJndiName">java:comp/env/{DS NAME}</property>
  

Tuesday, September 23, 2014

Lucene + Hibernate criteria : A hybrid approach to create effective filtering of results

Apache Lucene

Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.
Apache Lucene is an open source project available for free download. Please use the links on the right to access Lucene.

Hibernate Search
Built on top of Apache Lucene Hibernate search offers full-text search support for objects stored by Hibernate ORM, Infinispan and other sources. Think of it as Google for your entities:


  • search words with text
  • order results by relevance
  • find by approximation (fuzzy search)


One day I received a requirement, where-in I was supposed to implement fuzzy search ability in my application. My immediate thought is to implement hibernate-search as the project was already making use of hibernate as ORM. But then I came across difficulties which appeared to be hurdles in implementation.

I had many entities in my application with soft delete mechanisms "status = deleted" and then subscription date range, approval statuses "status = approved" and all. Now such criterias started becoming a big pain as I was newbe in Lucene and Hibernate-Search technologies. I started hunting internet to find an approach where I should be able to use words/phrases/sentences and also apply these default filters on my entities so as to fetch only those entities containing passed words/phrases/sentences and which are fallen within a given date range/status and all.

Finally after lot of hunt on internet and a very kewl hibernate-search documentation I was able to find a hybrid approach where I will be able to create a Lucene query along with hibernate criteria to filter out results. Here I am trying to explain the same thing.

Assume having a following entity structure

@Entity
@Table(name="organization")//Db table
@Indexed(index="Organization")//Physical directory name on file system path can be mentioned in hibernate property file
//Custom analyzer to provide custom tokanization for phonetic search ability
@AnalyzerDef(name="customanalyzer",
  tokenizer=@TokenizerDef(factory = StandardTokenizerFactory.class),
  filters={
   @TokenFilterDef(factory = LowerCaseFilterFactory.class),
   @TokenFilterDef(factory = PhoneticFilterFactory.class,
    params=@Parameter(name = "encoder", value = "DoubleMetaphone")
   )
}
)
public class ShodOrganization implements Serializable {
 private static final long serialVersionUID = 1L;

 @Id
 @Column(unique=true, nullable=false)
 @GeneratedValue
 private Integer organizationId;

 @Column
 @Field(index=Index.YES,store=Store.YES,analyze=Analyze.YES)
 @Analyzer(definition="customanalyzer")
 private String address;

 @Column
 @Field(index=Index.YES,store=Store.YES,analyze=Analyze.YES)
 @Analyzer(definition="customanalyzer")
 private String city;

 @Column
 private String country;

 @Column
 private String description;

 @Column(nullable=false)
 private Integer isApproved;

 @Column(nullable=false)
 private Integer enabled;

 @Column(nullable=false)
 @Field(index=Index.YES,store=Store.YES,analyze=Analyze.YES)
 @Analyzer(definition="customanalyzer")
 private String organizationName;

 @Column
 @Field(index=Index.YES,store=Store.YES,analyze=Analyze.YES)
 @Analyzer(definition="customanalyzer")
 private String pin;

 @Column
 @Field(index=Index.YES,store=Store.YES,analyze=Analyze.YES)
 @Analyzer(definition="customanalyzer")
 private String state;

 @Column
 private String email;
 
 @Column
 private String website;
 


 public ShodOrganization() {
 }

 public Integer getOrganizationId() {
  return this.organizationId;
 }

 public void setOrganizationId(Integer organizationId) {
  this.organizationId = organizationId;
 }

 public String getAddress() {
  return this.address;
 }

 public void setAddress(String address) {
  this.address = address;
 }

 public String getCity() {
  return this.city;
 }

 public void setCity(String city) {
  this.city = city;
 }

 public String getCountry() {
  return this.country;
 }

 public void setCountry(String country) {
  this.country = country;
 }

 public String getDescription() {
  return this.description;
 }

 public void setDescription(String description) {
  this.description = description;
 }

 public Integer getIsApproved() {
  return this.isApproved;
 }

 public void setIsApproved(Integer isApproved) {
  this.isApproved = isApproved;
 }

 public String getOrganizationName() {
  return this.organizationName;
 }

 public void setOrganizationName(String organizationName) {
  this.organizationName = organizationName;
 }

 public String getPin() {
  return this.pin;
 }

 public void setPin(String pin) {
  this.pin = pin;
 }

 public String getState() {
  return this.state;
 }

 public void setState(String state) {
  this.state = state;
 }

 public Integer getEnabled() {
  return enabled;
 }

 public void setEnabled(Integer enabled) {
  this.enabled = enabled;
 }

 public String getEmail() {
  return email;
 }

 public void setEmail(String email) {
  this.email = email;
 }
}

//Following API can be put in DAO implementation class in my case it was OrganizationDAOImpl
private Criteria getCriteria(String[]selectedCategories){
  Criteria criteria = currentSession().createCriteria(Organization.class);
  criteria.add(Restrictions.eq("isApproved", Constants.APPROVESTATUS.APPROVE.getValue()));
  criteria.add(Restrictions.eq("enabled", Constants.ENABLESTATUS.ENABLED.getValue()));
  return criteria;
  
 }

//Following API can be put again in same or a special Generic class for more Generic implementation



/**
* t : Type of class on which search needs to performed
* queryString : word or phrase which needs to be searched with full text search ability
* pageNo,pageSize for pagination
* criteria: hibernate criteria obtained from above method
*/
public List getSearchResults(Class t,String queryString,
   Integer pageNo,Integer pageSize, Criteria criteria){
  LOGGER.debug("Getting results for "+t.getName());
  if(null == queryString || queryString.isEmpty()){
   return getSearchResults(criteria, pageNo, pageSize);
  }
  Session session = getSession();//Get hibernate session from sessionfactory
  String [] searchFields = new String {"organizationName","address","city","state","pin","description"};
  FullTextSession fullTextSession = Search.getFullTextSession(session);
  org.apache.lucene.search.Query query = buildQuery(queryString, searchFields, fullTextSession, t);
  FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(query,t);
  if(null != criteria){
   fullTextQuery.setCriteriaQuery(criteria);
  }
  //Paginate
  if(pageNo == null || pageNo == 0){
   pageNo = 1;
  }
  if(pageSize == null || pageSize == 0){
   pageSize = 5;
  }
  Integer rec = pageNo*pageSize - (pageSize -1);
  fullTextQuery.setFirstResult(rec-1);//Start from pageno.
  fullTextQuery.setMaxResults(pageSize);//Return this much records
  LOGGER.debug("search query is "+query.toString());
  if(null != criteria){
   LOGGER.debug("criteria is "+ criteria.toString());
  }
  LOGGER.debug("records number "+rec+" pagesize is "+pageSize);
  return fullTextQuery.list();
  
 }
private org.apache.lucene.search.Query buildQuery(String queryString,String [] searchFields,FullTextSession fullTextSession,Class t ){
  if(null != queryString && !StringUtils.isEmpty(queryString) && StringUtils.contains(queryString, ' ')){
   LOGGER.info("creating phrase query");
   BooleanQuery bq = new BooleanQuery();
   for(String searchField : searchFields){
    PhraseQuery query = new PhraseQuery();
    StringTokenizer st = new StringTokenizer(queryString, " ");
     if(null != st){
      while(st.hasMoreTokens()){
       query.add(new Term(searchField,st.nextToken()));
      }
      bq.add(query, Occur.SHOULD);
     }
    }
   return bq;
  }
  
  QueryBuilder qb = fullTextSession.getSearchFactory().buildQueryBuilder().forEntity(t).get();
  return qb.keyword().onFields(searchFields).matching(queryString).createQuery();
}
Using above way you should be able to use hibernate criteria along with lucene text search ability to implement a state-of-art search in your application

Tuesday, August 19, 2014

Spring MVC + BIRT + Scripted Data Source

One fine morning I was looking for birt report implementation where in alike jasper the framework should be able to receive a populated collection/s of POJOs and render the same in report.

The quest gave me a real hard time to find an effective and efficient solution and finally yoooooo..EUREKA!!!!!!!!!!!!!

The solution was using a scriped data source in BIRT. But after finding a lil light in a whole blackhole darkness finally I found something which was relevant. I was happy but woooo then I realized it was actually sign of a coming trouble.
I was already running on a fire fighting mode and pressure situation, albeit I was able to find out a way of using scripted data source to populate a collection/s of POJOs to render the same on report I lost another day to find how to do this task or more specifically a step through example of this.

After questing a lot on google I was finally able to club some code from different articles/documentations to implement a final solution. And that's the reason I started writting this out to help somebody pulling hair to find such solution on web.

Here are the steps to use scripted data source and exploit spring's service layer to populate POJOs and render a report.

Assuming the reader has a background of spring mvc, BIRT, and has BIRT installation in existing web application & BIRT eclipse report designer.

1. Create a Simple Java classes.

POJOs
public class Employee {
 
 private String name;
 private String dob;
 private String address;
 private String mobile;
 public String getName() {
  return name;
 }
 public void setName(String name) {
  this.name = name;
 }
 public String getDob() {
  return dob;
 }
 public void setDob(String dob) {
  this.dob = dob;
 }
 public String getAddress() {
  return address;
 }
 public void setAddress(String address) {
  this.address = address;
 }
 public String getMobile() {
  return mobile;
 }
 public void setMobile(String mobile) {
  this.mobile = mobile;
 }

}
     



Service:



public interface EmployeeService extends GenericService{

   List getReportObjects(String departmentId);

}

@Service //Spring bean configuration annotation
public class EmployeeServiceImpl extends GenericServiceImpl implements EmployeeService {

  public List getReportObjects(String departmentId){
  System.out.printLn("Department Id is just how to pass a parameter from report to spring layer to filter results "+departmentId);
  List list = new ArrayList();
  
  Employee e = new Employee();
  e.setName("anant");
  e.setDob("26-Jun-1982");
  e.setAddress("my Address 1");
  e.setMobile("9912345667");
  list.add(e);
  e = new Employee();
  e.setName("Ashay");
  e.setDob("26-Jul-1984");
  e.setAddress("my Address 1");
  e.setMobile("9912345667");
  list.add(e);
  e = new Employee();
  e.setName("anant");
  e.setDob("26-Jun-1982");
  e.setAddress("my Address 1");
  e.setMobile("9912345667");
  list.add(e);
  return list;
 }
}



2. Use eclipse birt plugin to create a report design file test.rptdesign
3. Under "Data Explorer ---> Data Sources --> Create New Data Source ---> Scripted Data Source ---> Finish".


4. Under "Data Explorer ---> Data Sets --> Create New Data Set ---> Next --->".
5. On next screen of data set creation wizard Click on "Add" you need to create number of output columns this data set should have.
6. "Column Name" element and "Display Name" element are the key element. String which you will enter in "Column Name" will be used to later in article to store data in data set elements.

7. Select ""Data Explorer ---> Data Sets" and select "script" tab.
8. Under "script --> open" use following Rhino script to get spring application context instance and an instance of you service class bean.


importPackage(Packages.org.springframework.context);
importPackage(Packages.org.springframework.web.context.support );
//ServletContext
var sc = reportContext.getHttpServletRequest().getSession().getServletContext();
//ApplicationContext 
var spring = WebApplicationContextUtils.getWebApplicationContext(sc);
//your spring bean 
var employeeService = spring.getBean("employeeService");
//Load the rows (Note here is where you are able to pass your parameter into the Java layer)

//Get employee list from your spring service bean for a given department Id
var data = employeeService.getReportObjects(params["departmentId"]);

dataIterator = data.iterator();


9. You must be wondering what is "params["departmentId"]"? This is a report parameter which will be created under "Data Explorer --> report Parameters" and will be entered by user.
10. Under "script --> fetch" use following Rhino script to fetch and assign data in data set.

 if(dataIterator.hasNext()){
 var object = dataIterator.next();
 //name,dob,address,mobile will be "Column Name" in output columns window in data set creation wizard as discussed earlier
 // Please do not refer display names
    row["name"] = object.getName(); //Getter of Employee Domain object
    row["dob"] = object.getDob();
    row["address"] = object.getAddress();
    row["mobile"] = object.getMobile();
    return true;
 }else{
  return false;
 }

11. Drag your data set on report layout and you are good to go.
12. All remains is a beutification part and I am very poor in color selections and can not help you out there

Thursday, July 24, 2014

Hibernate appends LIMIT in every query automatically

Once in a morning I woke up with a very special issue in my web application and by afternoon it almost made me to pull out my hair.
I am using Spring hibernateDAOSupport at DAO layer of my application and am using hibernateTemplate with detachedcriteria to fetch records from my DB. This time I was assigned an intermittent issue where in the query API in DAO was not able to fetch correct number of records from my DB.
I always like challenges when it comes to, and I always need a couple of coffee mugs to get in DEBUG mode.

While debugging I jumped right away on the hibernate generated SQL where I was able to trace my issue. The issue was hibernate was appending "LIMIT ?" at end of my query even though I do not intended to do this.



    DetachedCriteria detachedCriteria = DetachedCriteria.forClass(ABC.class);
    detachedCriteria.add(Restrictions.in("col1", phone));
    detachedCriteria.add(Restrictions.isNotNull("col2"));
    list = getHibernateTemplate().findByCriteria(detachedCriteria);


Do you see any setMaXResults statement? Well I also don't as like you. Then why the ??? hibernate appending "LIMIT ?" these two words as add-on and making me smash my head on wall.

Wooooooooo but I stopped there and thought Hibernate Spring can not be blamed for this it should be something which I am doing. and now I at least know where am I going wrong. So lets find solution.

Solution:

Well after certain Google try I found some links, blogs people discussing about this. Now here are some work around for the same.

1. A quick hack is to reset maxResults to null or 0 when you use setMaxResults to fetch limited records from DB

2. Another is bit wordier but a nice approach where you need not to rest max results everytime


    protected List getWrappedStuff() {
    List results = new ArrayList();
    HibernateTemplate hibernateTemplate = getHibernateTemplate();
    hibernateTemplate.setMaxResults(10);

    StringBuilder hsql = new StringBuilder("select * from table1");

    List queryResults = hibernateTemplate.find(hsql.toString());
    for (Object[] result : queryResults) {
        results.add(new myobject((String)result[0], (Long)result[1]));
    }

    return results; 
}
The question really is, is it a nice practice to use HibernateTemplate and how can we use this

All spring templates (hibernate, jdbc, rest, jpa etc.) have the same pros and cons:

Pro: They perform common setup routines for you, let you skip the boilerplate and concentrate on the logic you want.

Con: you are coupling your application tightly to the spring framework. For this reason, Spring recommends that HibernateTemplate no longer be used.

Specifically, what HibernateTemplate did for you was to automatically open and close sessions and commit or rollback transactions after your code executed. However, all of this can be achieved in an aspect-oriented way using Spring's Declarative Transaction Management.

So time for you to think.

Monday, July 21, 2014

Spring MVC serving images from physical file system

Spring MVC serving images from physical file system

Very offen people are in quest of a good solution to store and potray user images viz. personal images, signatures etc and like many people I found myself as one looking out for this kinda solution googling, binging all around internet with almost no notion.

Well, but "Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better."
One nice morning I was able to find a solution for this using Java Spring.

Spring gives you a very nice way to handle how you would display images which are stored somewhere outside your web server/servlet container sandbox.

This tutorial will explain how you should configure your Spring MVC application if you want to serve resources that reside outside the webapp directory, or even outside the whole project archive. In short, how to access files from anywhere in the file system with your static resources.
You need to configure access to your static resource by putting the following line in the {servlet-context.xml} file:



Serving resources from outside the context root
Now, this is accomplished by specifying an absolute path to the resource folder in the location attribute of the resources tag:



NOTE: don't forget to put the trailing slash at the end of the absolute path. It won't work if you miss that.
Serving resources from outside the context root - Windows path



Hope this will spare you some sleepless hours, because it took me a while to figure it out myself.

After context file configuration you should be able to serve images using a simple image tag.
viz.

Good Luck

Friday, July 18, 2014

Resolving JAR hell The Hack

Resolving JAR hell The Hack

I never knew about how Java class loader works. Frankly speaking just read this line while interview preparation.

When explored I came to know how it really works, how bootstrap/system classpath makes a difference, how web app lib directory creates a significant difference and all.




Actually there is a way to solve this and this is using your custom class loader but well sometimes you run out of time when you have a crunched time and a critical delivery ahead and that's where I was standing.


I had a situation to load "XSSFWorkbook" in my web application to read data from excel sheet and had two JARS "poi-ooxml-3.9.jar" and "org.eclipse.birt.runtime_4.3.1.v20130918-1142.jar" with same class name and package structure. I simply remaned the later so in its alphabetical order comes next to the earlier one and it did a trick for me. Just wanted to share the same.

Hope this will help somebody from some sleepless nights and crunched situation


Well and curious to mention that this is my very first post so apologize for incorrect English comprehension and excited to see you all sharing knowledge.

Good Luck