FOREIGN KEYS just ensure your data are consistent.
They do not improve queries in sense of efficiency, they just make some wrong queries fail.
If you have a relationship like this:
CREATE TABLE department (id INT NOT NULL)
CREATE TABLE employee (id INT NOT NULL, dept_id INT NOT NULL, FOREIGN KEY (dept_id) REFERENCES department(id))
, then you cannot delete a department if it has some employee's.
If you supply ON DELETE CASCADE to the FOREIGN KEY definition, the referencing rows will be deleted automatically along with the referenced ones.
As a constraint, FOREIGN KEY actually slows down the queries a little.
Extra checking needs to be performed when deleting from a referenced table or inserting into a referencing one.