It is safe because nothing is created during the swap operation. Only data members of the class std::vector are swapped.
Consider the following demonstrative program that makes it clear how objects of the class std::vector are swapped.
#include <iostream>
#include <utility>
#include <iterator>
#include <algorithm>
#include <numeric>
class A
{
public:
    explicit A( size_t n ) : ptr( new int[n]() ), n( n )
    {
        std::iota( ptr, ptr + n, 0 );   
    }
    ~A() 
    { 
        delete []ptr; 
    }
    void swap( A & a ) noexcept
    {
        std::swap( ptr, a.ptr );
        std::swap( n, a.n );
    }
    friend std::ostream & operator <<( std::ostream &os, const A &a )
    {
        std::copy( a.ptr, a.ptr + a.n, std::ostream_iterator<int>( os, " " ) );
        return os;
    }
private:    
    int *ptr;
    size_t n;
};
int main() 
{
    A a1( 10 );
    A a2( 5 );
    std::cout << a1 << '\n';
    std::cout << a2 << '\n';
    std::cout << '\n';
    a1.swap( a2 );
    std::cout << a1 << '\n';
    std::cout << a2 << '\n';
    std::cout << '\n';
    return 0;
}
The program output is
0 1 2 3 4 5 6 7 8 9 
0 1 2 3 4 
0 1 2 3 4 
0 1 2 3 4 5 6 7 8 9 
As you see only data members ptr and n are swapped in the member function swap. Neither additional resources are used.
A similar approach is used in the class std::vector.
As for this example
std::vector<Widget> WidgetVector;
std::vector<Widget2> Widget2Vector;
then there are objects of different classes. The member function swap is applied to vectors of the same type.