function(a) means, call the function function with the argument a.
function([a]) means, call the function with the list [a] as its argument. Equivalently, you could write function(list(a))
Both characters have other uses; round parentheses can be used to group expressions, like in mathematics, which in Python also gets used to indicate a tuple. So,
function((a,)) means, call the function with the tuple (a,) as the argument. The comma is necessary to make it into a tuple; just (a) is merely the mathematical grouping we mentioned before, saying evaluate a before ... nothing. Equivalently, you could write function(tuple(a))
Square brackets are also used in indexing, so listvar[a] means the a:th element of the list variable listvar, and dictvar[a] means get the value for the key a from the dictionary dictvar.
For lists, the notation actually allows you to pull out sublists, called slices, in various ways, like listvar[:-a] or listvar[::]. This is complex enough that I'll defer the explanation of it to a separate question: Understanding slice notation
Square brackets are also used for various matrix notations in Numpy and Pandas, but that's not part of the Python core.