Query slows down by a factor of 250 to 450 when increasing maximum server memory setting Why ?

We have an SQL server 2012 SP1 CU6 running on a virtual machine that was P2V'ed from a physical blade server.

configured from the physical server was maintained hence it is still configured with 4 CPU's and 6 Cores hence having access to 24 cores in total.
The physical server had 128 gb.

The host (DL580G7, 4 cpu's with 10 cores, 1 TB memory) is running ESXI 5.1

All job are running slightly better on the VM due the better hardware.

Now the funny part starts. After doing the initial testing with the machine configuration which was all good we wanted to increase the VM memory to 512 gb. as we have a total of 1 TB of memory in our hosts and no memory pressure.

We configured the maximum server memory to 80% of the 512 gb. memory and then some of our queries starting taking ages.
We tweaked and adjusted the maximum server memory and found out if we configured it to 330 gb. then server runs just fine but if we go above say 335 gb. then the following
job.:

USE Datamart
exec sp_executesql N'set dateformat dmy exec ssis.tblSumOfStock @P1',N'@P1 datetime2(0)','2013-01-31 00:00:00'
GO

Normally the job takes between 76 and 77 sec. but with the increased memory setting it has taken between19960 and 35254 seconds (roughly 5h 30m - 9h 45m). and if max memory is lowered execution time return to 76 sec.

We have tried the following.

1.: Reserving all memory on the host
2.: Running sql server with flag 0815 and 8048 to see if numa was the cause but with 330 gb. is still goes across several numa nodes.
3.: Tried isolation the VM on two different hosts - Same issue.
4.: Tried SQL flag to aquirie all memory on startup.
5.: MAXDOP -1 job still very slow
6.: Increasing parallel cost to different values - again slow run
7.: Comparing execution plans with the different memory settings does not give any clues to why.

I simply cant figure when this is happening. Have any of you noticed anything like this or know what could cause this ?

When running the job with max memory setting below and above the 330 gb mark the server acquires a total of 60 gb memory hence there is no memory pressure.