First, you mixed mysql_* with mysqli_* extension. It would be advisable to just use mysqli_* rather than the deprecated mysql_*.
Assuming that you establish your connection to your database using mysqli_* extension (db_connect.php):
$conn = new mysqli("Host", "Username", "Password", "Database"); /* REPLACE NECESSARY PARAMETERS */
/* CHECK CONNECTION */
if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
    exit();
}
Then your main file:
<?php 
    include ('db_connect.php');
    echo '<select name="ins_name">';
    $stmt = $con->prepare("SELECT ins_name FROM institution"); /* PREPARE QUERY */
    $stmt->execute(); /* EXECUTE QUERY */
    $stmt->bind_result($insname); /* BIND RESULT TO THIS VARIABLE */
    while($stmt->fetch()){ /* GET ALL RESULT */
        echo '<option value="'.$insname.'">'.$insname.'</option>'; 
    } /* END OF WHILE LOOP */
    $stmt->close(); /* CLOSE STATEMENT */
    echo '</select>';
?>
And also, you did not set a data to display inside your <option> in your example. You can see above how we concat the value in your <option></option>