This is a question I was asked today and this is a question people ask often.
The answer is NO in most of the situations if it is used merely to keep goods that are purchased in Japan to ship abroad.
The Japan-US tax treaty Article #9 says that a warehouse can be a PE along with other situations such as a branch, an office, a factory, or an exclusive agent.
But the article also mentions the exceptions as following:
A permanent establishment shall not include a fixed
place of business used only for one or more of the following;
(a) The use of facilities for the purpose of storage, display, or delivery of goods or merchandise
belonging to the resident;
(b) The maintenance of a stock of goods or merchandise belonging to the resident for the purpose
of storage, display, or delivery;
(c) The maintenance of a stock of goods or merchandise belonging to the resident for the purpose
of processing by another person;
(d) The purchase of goods or merchandise, or the collection of information, for the resident;
A Permanent Establish usually means a fixed place of business. A warehouse that only holds purchased merchandise to be shipped abroad is apparently not considered as a full business activity.