Limits on U.S. imports, especially for autos, will hamper growth, a port executive told CNBC on Friday.