GPLv3
For the GPLv3, no, you do not have the right to distribute the work.
The GPLv3 allows the distribution of a GPL work without the normal full freedoms, in the narrow case that the distribution is done explicitly for the recipient to make modifications. The exact language is in section 2, which applies to contractors who modify the software and to remote execution environments (e.g., "cloud" providers like Amazon Web Services do not get GPL rights when you give them a copy of GPLv3 software to run):
You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you.
The rationale for this change was addressed in the release notes for the GPLv3 final discussion draft:
4 Conveying to Outside Contractors
Large enterprise users of free software often contract with non-employee
developers, often working offsite, to make modifications intended for the
user’s private or internal use, and often arrange with other companies to
operate their data centers. Whether GPLv2 permits these activities is not
clear and may depend on variations in copyright law. The practices seem
basically harmless, so we have decided to make it clear they are permitted.
GPLv3 now gives an explicit permission for a client to provide a copy of
its modified software to a contractor exclusively for that contractor to modify
it further, or run it, on behalf of the client. However, the client can only
exercise this control over its own copyrighted changes to the GPL-covered
program. The parts of the program it obtained from other contributors must
be provided to the contractor with the usual GPL freedoms.
This permission is stated in section 2. It permits a user to convey covered
works to contractors operating exclusively on the user’s behalf, under the
user’s direction and control, and to require the contractors to keep the user’s
copyrighted changes confidential, but only if the contractor is limited to
acting on the user’s behalf, just as the user’s employees would have to act.
The strict conditions in this provision are needed so that it cannot be
twisted to fit other activities, such as making a program available to users or
customers. By making the limits on this provision very narrow, we ensure
that in all other cases the contractor gets the full freedoms of the GPL.
GPLv2
As the above release note indicates, the situation is complex for GPLv2. The situation as it exists outside the GPLv3's special excpetion is addressed in the GPL FAQ (take note of both paragraphs):
Is making and using multiple copies within one organization or company "distribution"?
No, in that case the organization is just making the copies for itself. As a consequence, a company or other organization can develop a modified version and install that version through its own facilities, without giving the staff permission to release that modified version to outsiders.
However, when the organization transfers copies to other organizations or individuals, that is distribution. In particular, providing copies to contractors for use off-site is distribution.
If this client is distributing someone else's GPL-licensed work (or a derivative) they are only permitted to do so under the GPL. Does your case qualify as distribution? This FAQ item certainly presents the unambiguous opinion that off-site distribution to a contractor (or any non-employee) constitutes distribution. This basically reduces to the question of "Are you an employee or a contractor?" If you're an employee, you are an agent of the company, so the code isn't moving outside the company's possession. If you're a contractor, then your receipt of the code is external to the perimeter of the company, and it is distribution.
In case you're not convinced, this case is very unambiguously covered in another FAQ item (note the final paragraph, which I have emphasized):
Does the GPL allow me to develop a modified version under a nondisclosure agreement?
Yes. For instance, you can accept a contract to develop changes and agree not to release your changes until the client says ok. This is permitted because in this case no GPL-covered code is being distributed under an NDA.
You can also release your changes to the client under the GPL, but agree not to release them to anyone else unless the client says ok. In this case, too, no GPL-covered code is being distributed under an NDA, or under any additional restrictions.
The GPL would give the client the right to redistribute your version. In this scenario, the client will probably choose not to exercise that right, but does have the right.
So, your client can place a restriction in your contract about when you may (or may not) release your own changes, but they cannot legally tell you when you may or may not release their code that they distributed to you under the GPL.
Of course, even if there is no legal impediment to distributing your client's code without their say-so, that sounds like a very efficient way to demolish your trustworthiness as a contractor. They can't sue you, but they are free to hire someone else for future work.