Is there an easier way to do so?
$array = array(1,57,5,84,21,8,4,2,8,3,4);
$remove = 21;   
$i = 0;
foreach ($array as $value ){
    if( $value == $remove)
        unset($array[$i])
        $i++;
    }
//array: 1,57,5,84,8,4,2,8,3,4
Is there an easier way to do so?
$array = array(1,57,5,84,21,8,4,2,8,3,4);
$remove = 21;   
$i = 0;
foreach ($array as $value ){
    if( $value == $remove)
        unset($array[$i])
        $i++;
    }
//array: 1,57,5,84,8,4,2,8,3,4
 
    
    The array_search answer is good.  You could also arraydiff like this
$array = array(1,57,5,84,21,8,4,2,8,3,4);
$remove = array(21);
$result = array_diff($array, $remove); 
 
    
    If you want to delete the first occurrence of the item in the array, use array_search to find the index of the item in the array rather than rolling your own loop.
$array = array(1,57,5,84,21,8,4,2,8,3,4);
$remove = 21;
$index = array_search($remove, $array);
if (index !== false)
  unset($array[$index]);
To remove all duplicates, rerun the search/delete so long as it finds a match:
while (false !== ($index = array_search($remove, $array))) {
  unset($array[$index]);
}
or find all keys for matching values and remove them:
foreach (array_keys($array, $remove) as $key) {
  unset($array[$key]);
}
 
    
    This is a little cleaner:
foreach($array as $key => $value) {
    if ($value == $remove) {
        unset($array[$key]);
        break;
    }
}
UPDATE
Alternatively, you can place the non-matching values into a temp array, then reset the original.
$temp = array();
foreach($array as $key => $value) {
    if ($value != $remove) {
        $temp[$key] = $value;
    }
}
$array = $temp;
