Where do I and the Contractor stand with the agency.

The IT contractor is on a 6 month contract with an IT agency. This contract has just expired and I would like the contractor to contract direct with me. we have used this contractor for 3 years now via the agency and feel it is time to offer a direct IT contract on a per project basis. Will either of us be taken to court?
Denise Hudson